Language models are traditionally evaluated with perplexity.
we tackle these three issues: we propose an ASRmotivated evaluation setup which is decoupled from an ASR system and the choice of vocabulary, and provide an evaluation dataset for English-Spanish code-switching.