However, the decoding speed is significantly worse since the decoder needs to apply K NMT models rather than only one.
a recent line of research transfers the idea of knowledge distillation (Bucilu et al., 2006;Hinton et al., 2014) to NMT and trains a smaller network (the student) by minimizing the cross-entropy to the output of the ensemble system (the teacher) (Kim and Rush, 2016;Freitag et al., 2017).