In the bottom, a bidirectional LSTM layer encodes the input sentence to extract non-local features.
we append a starting symbol <s> and an ending symbol </s> to the left-to-right LSTM and the right-to-left LSTM, respectively.