As can be seen from Table 3, alternative word generation orders all make it harder for standard leftto-right sequence decoder to learn to predict the next word.
using a doublyrecurrent architecture, specifically the surface decoder and the ancestor decoder, can improve perplexity scores for top-down word generation over the left-to-right decoder.