The problem with these approaches is that the modeler has to make design decisions about how to encapsulate contextual information as network inputs.
our approach here is to simply provide the entire conversation history as input to a standard LSTM-LM, and let the network learn the information that is relevant to next-word prediction.