From this benchmarking table, it can be observed that the attention mechanism is helpful for decoders to improve the relevance of the generated responses, since the Attention-Seq2Seq performs better than the basic Seq2Seq on the dataset, in terms of all the three metrics.
the relative gain of the attention layer is limited, indicating that modeling relation of query and response by attention module is not able to directly solve the learning paradigm of conversations.