Although attention is an intuitively appealing concept and has been proven in practice, existing models of attention use contentbased addressing and have made only limited use of the historical attention masks.
lessons from better word alignment priors in latent variable translation model suggests value for modeling attention dependent of content.