In order to evaluate how cross-lingual structural variation hinders the design of effective generalpurpose algorithms, we propose the task of language modeling (LM) as a testbed.
we opt for a full-vocabulary setup where no word encountered at training time is treated as an unknown symbol, in order to a) ensure a fair comparison across languages with different word frequency rates and b) avoid setting an arbitrary threshold on vocabulary size (Cotterell et al., 2018).