According to the structure shown in Figure 1, W 1 is a matrix of weights that is updated during training, thus the distributed representations contained in W 1 are learned simultaneously with the training of BLSTM-RNN on any supervised learning tasks.
recently, many state-of-the-art systems of tagging related tasks are implemented with bidirectional long short-term memory (BLSTM) recurrent neural network (rNN), for example, slot filling (Mesnil et al., 2013), part-of-speech tagging (Huang et al., 2015), and dependency parsing (Dyer et al., 2015) etc.