DU has the form of , in which w i , w i+1 are two adjacent tokens and d ii+1 is the dependency between them.
the low-dimensional forward and backward representation vectors of DU j are created by concatenating the corresponding final embeddings of tokens w j , w j+1 and the LSTM hidden state of the dependency d ii+1 .