Automatic and human evaluations show that an abstractive model trained with a multi-task objective outperforms conventional Seq2Seq, language modeling, as well as a strong extractive baseline.
our best model is still far from human performance since raters prefer gold responses in over 86% of cases, leaving ample opportunity for future improvement.