Multilingual pretraining of languages from a different language family and script benefitted from an explicit phoneme objective and adversarial objective when there was sufficient diversity in the pretraining languages.
a change in orthography was conflated with a change in language family, geographic location, and phono- 14.8 (-6.3%) Avg.