The concatenated representation of bi-directional hidden states: , is regarded as the encoded vector for each input word.
finally, the input sequence is transformed into a hidden state sequence: the initiated hidden state is a zero vector, and the hidden state (h r L i ) in the last word could be used for initiating the next encoder (lowlevel encoder).