Feeding the next target token assumes that we know it in advance and thus calls for separate translation and alignment models.
we train the layer average baseline with a batch size of 7168 tokens on 64 Volta GPUs for 30k updates and apply a learning rate of 1e-3, β 1 = 0.9, β 2 = 0.98.