The preceding utterances are serialized in the order of their onset times, so that the flow of words within an utterance is not disrupted.
all LSTM-LMs are pretrained for one or two epochs on a large corpus of "conversational Web" data (Bulyko et al., 2003), followed by normal training to convergence on the in-domain data.