To model long-term dependencies, an attention module is applied to summarize all previous contexts into a global context vector.
a feed-forward network takes the current context RNN's output c t as the query and all previous context outputs from both context RNNs as keys and values to compute attention scores.