Recent work on interpretability for NLU tasks uses methods such as diagnostic tasks (Belinkov et al., 2017;Tenney et al., 2019;, attention based methods (Raganato and Tiedemann, 2018) or task analysis (Zhang and Bowman, 2018) and is primarily focused on understanding the linguistic features encoded by a trained model.
due to the differing nature of translating multiple languages to English and vice versa, the representation space of these two sets of languages, All-to-English and English-to-Any, behave differently and separate quite clearly ( Figure 11 in the Appendix).