The word-level abstraction summarizes information from each previous sentence j into a vector s j as: where h denotes a hidden state of the NMT network.
h t is the last hidden state of the word to be encoded, or decoded at time step t, and h j i is the last hidden state of the i-th word of the j-th sentence of the context.