As is common for CNNs and even recent RNN term encodings (Peters et al., 2018), we use the original term embedding e(t i ) as a residual and combine it with the BILSTM encodings.
if are the last layer's hidden states of the left-to-right and right-to-left LSTMs for term t i , respectively, then we set the context-sensitive term encoding as: Since we are adding the original term embedding to each LSTM hidden state, we require the dimensionality of the hidden layers to be equal to that of the original embedding.