Parameters h j and H j are the hidden vectors at the j th step of the wordlevel and sentence-level decoder respectively defined as: where The non-linear transformation, φ (we choose tanh), is used to connect the word-level encodings to the sentence decoder and the sentence-level encodings to the word decoder.
the word-level decoder updates its state by considering a sum of sentence encodings, weighted by the attentions from the previous state and mutatis mutandis for the sentence-level decoder.