In the NLP literature, model-based reinforcement learning techniques are commonly used for dialog management (Singh et al., 2002;Lemon and Konstas, 2009;Schatzmann and Young, 2009).
learning directly from annotations results in a low-quality environment model due to the relative lack of exploration,  Figure 4: Examples of automatically generated paraphrases for high-level instructions.