We note that a model with an observed |µ z | >> 0 on a dev set, can be 'corrected' to a large extent (as we show later) by subtracting this dev µ z from the unnormalized scores at test time.
we investigated the following language modeling methods: • DEV-LM -the language model proposed by Devlin et al.