One may expect that the gaps between the different models should become smaller and even close with more updates, as the models with more languages see less examples per language in each batch, thus requiring more updates to improve in terms of BLEU.
in our setting these gaps did not close even after the models converged, leaving 2.73 average BLEU difference be-tween the 5-to-5 and the 103-to-103 model.