(a) Baseline model with a single-layer decoder RNN and attention (b) Addition of a source-independent LM layer that feeds into the source-dependent decoder (c) Multi-task setting next-word prediction from both layers; green softmax layers are shared.
the use of synthetic parallel data still outperforms our approach both in single and ensemble systems.