A long list of objective dialogue metrics (Danieli and Gerbino 1995;Smith and Gordon 1997) for dialogue evaluation, which can be calculated without recourse to human judgment, and subjective dialogue metrics (Shriberg, Wade, and Price 1992;Danieli and Gerbino 1995), which are based on human judgments, have been proposed.
while the task of the wizard in the first WOZ experiment was to perform speech understanding and dialogue management, the task in the second WOZ experiment was to perform only speech understanding, and the dialogue-management task was assigned to the newly implemented dialogue manager.