While other work has sought to address this problem (Cohn et al., 2016;Tu et al., 2016;Feng et al., 2016), these models either rely on explicitly engineered features (Cohn et al., 2016), resort to indirect modeling of the previous attention decisions as by looking at the content-based RNN states that generated them (Tu et al., 2016), or only model coverage rather than coverage together with ordering patterns (Feng et al., 2016).
we propose to model the sequences of attention levels for each word with an RNN, looking at a fixed window of previous alignment decisions.