Concatenate the forward and backward LM embeddings to form bidirectional LM embeddings, i.e.,  The embedding learned from BLSTM and LM, have various degrees of contribution to tag prediction.
concatenation fails to consider the imbalance between them; therefore, we aim to improve by using the attention mechanism (Rei et al., 2016).