Moreover, our transfer results are strong and homogeneous across all languages: For 11 of them, the zero-short performance  is (at most) 5% lower than the one on English, including distant languages like Arabic, Chinese, and Vietnamese, and we also achieve remarkable good results on low-resource languages like Swahili.
bERT achieves excellent results on English, outperforming our system by 7.5 points, but its transfer performance is much weaker.