Although the M may contain potential parallel sentences t for s, we cannot directly use (s, t ) as ground-truth sentence pairs to train the translation model V s→t because the NMT system is sensitive to noises (Cho et al., 2014a;Cheng et al., 2018).
note that (1) all the encoders share the same parameters (same for decoders); (2) the decoding processes are nondifferentiable, so the language modeling loss and the comparative translation loss are used to train the learning modules before and after the decoding processes, respectively.