During training, mentions from different languages are mixed using inverse-ratio mini-batch mixing strategy.
if two languages have training data sizes proportional to α : β, at any time during training, mini-batches seen from them are in the ratio 1 α : 1 β .