POMDPs need evaluation scores (i.e., d) for dialogue acts (i.e., a) for training a reward function.
we asked a third-party participant, who was neither a listener nor a speaker in our dialogue data collection, to evaluate the user satisfaction levels of the collected dialogues.