For word-level memory, we first apply a Bi-directional GRU (BiGRU) to compute the hidden states of all the ordered wordsw = {w t } t=(1,2,...,|t|) in the sentence set X, where |t| is the time steps of the words in the sentences.
for the t-th wordw t , the forward GRU and the backward GRU encode GRU (C Rw t ) respectively, where C R is the word embedding matrix of the last hop on the sentence-level memory, and we set the dimension of ⃗ h t and ← h t equals to the dimension of the word embedding.