In the context of Neural Machine Translation (NMT) this results in poor performance on heterogeneous datasets and on sub-tasks like rare phrase translation.
we use a Transformer based decoder that attends to both, the encoder outputs and the CSTM, in every cross-attention layer.