Results in Table 3 show that in all cases multilingual pretraining of QUE+CYR with the auxiliary objectives outperformed its counterpart without the objectives (which frequently undeperformed the monolingual model), and in all but one case this led to an improvement over the monolingual baseline.
7 To gain insight into how the auxiliary objectives change the representation of speech learnt by the 7 this doesn't hold in the language adaptation scenario, where the auxiliary objectives help QUE+CYR only slightly; see Section 8. models, we applied 2D t-SNE dimensionality reduction (Van Der Maaten and Hinton, 2008).