The problem with testing our policies against actual users is that there are a number of aspects of dialog that we have not modeled, such as nonunderstandings, misunderstandings, and even parsing sentences into the action specification and generating sentences from the action specification.
as a simplification we tested our learned system policy on the handcrafted baseline policies.