− log p(u gt |u 1 , ...q, ...u T ), where u gt is the original utterance at index t. The final loss function is: = − log exp(α gt ) MUR is analogous to the MLM objective of Devlin et al.
it is still unclear whether language modeling is the most effective method of pretrained language representation, especially for tasks that need to exploit multi-turn dependencies, e.g.