We use the same hyperparameters as in monolingual training but increase the batch size to 256 to account for the increase in the number of languages, and we use a = 0.7 as in Devlin et al.
the obvious approach to parsing ten languages is to fine-tune ten times, producing ten variants of the parameter-heavy BERT layers.