One of the most popular approaches for automatically evaluating the quality of dialogue responses is by computing their word overlap with the reference response.
the most popular metrics are the BLEU and METEOR scores used for machine translation, and the ROUGE score used for automatic summarization.