There is a big gap between this model's zeroshot performance and M-BERT's, showing the pretraining is helping learn a useful cross-lingual representation for grammar.
cross-script transfer is less accurate for other pairs, such as English and Japanese, indicating that M-BERT's multilingual representation is not able to generalize equally well in all cases.