Neural conversation systems, typically using sequence-to-sequence (seq2seq) models, are showing promising progress recently.
traditional seq2seq suffer from a severe weakness: during beam search decoding, they tend to rank universal replies at the top of the candidate list, resulting in the lack of diversity among candidate replies.