They report some improvements in translation quality arguing that the attention model has learned to better align source and target words.
in order to compare attentions of multiple systems as well as to measure the difference between attention and word alignment, we convert the hard word alignments into soft ones and use cross entropy between attention and soft alignment as a loss function.