While human evaluation should always be used to evaluate dialogue models, it is often too expensive and time-consuming to do this for every model specification (for example, for every combination of model hyperparameters).
having an accurate model that can evaluate dialogue response quality automatically -what could be considered an automatic Turing test -is critical in the quest for building human-like dialogue agents.