While separating out a language model allowed us to carry out multi-task training on mixed data types, it constrains gradients from monolingual data examples to a subset of source-independent network parameters (σ).
synthetic data always affects all network parameters (θ) and has a positive effect despite source sequences being noisy.