Nevertheless, the autoregressive property of the NMT decoder has been a bottleneck of the translation speed.
the decoder, whether based on Recurrent Neural Network (RNN) (Hochreiter and Schmidhuber, 1997;Cho et al., 2014) or attention mechanism (Vaswani et al., 2017), sequentially generates words.