The output of the is softmax activated to produce attention weights a [k] t at every stage k of the multistage fusion process: is fed as input into the HIGHLIGHT LSTM at stage k + 1.
the HIGHLIGHT LSTM functions as a decoder LSTM (Sutskever et al., 2014;Cho et al., 2014) in order to capture the dependencies on previous attention assignments.