Such user teaching precisely addresses Run π θ (a|s) with user to collect new dialogue samples D π Ask user to correct the mistakes in the tracked user's goal for each dialogue turn in D π Add the newly labeled dialogue samples to the existing corpora: Train model end-to-end on D and obtain an updated policy π θ (a|s) eq 4 7: end for 8: for learning iteration k = 1 : N do 9: Run π θ (a|s) with user for a new dialogue 10: Collect user feedback as reward r 11: Update model end-to-end and obtain an updated policy π θ (a|s) eq 5 12: end for the limitations of the currently learned dialogue model, as these newly collected dialogue samples are driven by the agent's own policy.
in this study we let an expert user to correct the mistake made by the agent in tracking the user's goal at the end of each dialogue turn.