To deal with this issue, we introduce knowledge distillation (Kim and Rush, 2016) to conduct the translation knowledge transfer.
during the transfer process from θ in , we first initialize θ in not only to match the references of D in , but also to be consistent with probability outputs of the previous best in-domain NMT model θ * in , which is considered as the teacher model.