Typically, evaluating goal-oriented dialogue systems is done via human-generated judgment like a task completion test or user satisfaction score (Walker et al., 1997;Möller et al., 2006).
the task of evaluating open-ended dialogue systems is not well defined as there is no * Equal Contribution clear explicit goal for conversations.