As mentioned in §1, using only the shared features for MLTL imposes an overly strong con-straint and many useful features may be wiped out by adversarial training if they are shared only between the target language and a subset of source languages.
we propose to use a mixture-of-experts (MoE) model (Shazeer et al., 2017;Gu et al., 2018) to learn the private features.