In practice, 4-layer QRNN encoder and decoder are used to gain translation quality that is comparable to that of singlelayer LSTM/GRU NMT.
our onelayer model achieves significantly higher performance than a 10-layer SRU system.