Even with an attention mechanism, the Seq2Seq model just pays attention to the current alignment information between the inputs and the output at the current position but ignores past alignments information.
it cannot keep track of the attention history when it updates information at each current time step, leading to the over-production (Tu et al., 2016a,c;Mi et al., 2016;Tu et al., 2016b).