The parameters of the network are trained to minimize the loss L task of the predicted and true distribu-tions on all the tasks as follows: where L(ŷ k , y k ) denote a typical cross-entropy loss for each task k. As overviewed in Section 1, the success of transfer learning depends on the sharing scheme in latent feature space.
we also report the results of XNLI baselines (Conneau et al., 2018), a supervised cross-lingual MTL model that combines the L adv loss using pseudoparallel data (Liu et al., 2017), the multilingual BERT (Devlin et al., 2018), and the recent work of (Artetxe and Schwenk, 2018).