Among various neural networks for sequence labeling, bi-directional RNNs (BiRNNs), especially BiLSTMs (Hochreiter and Schmidhuber, 1997) have become a dominant method on multiple benchmark datasets Chiu and Nichols, 2016;Lample et al., 2016;Peters et al., 2017).
there are several natural limitations of the BiLSTMs architecture.