This incongruity might become diffuse if the inputs are combined too soon, and we have shown that the multiple-LSTM architecture outperforms a single LSTM architecture that combines the current turn and the context.
lSTM networks with sentence-level attention achieved significant improvement when using prior turn as context for all the data sets (e.g., 6-11 percentage point F1 for IAC v2 and Twitter messages).