Finally, our work is inspired by techniques for improving upon maximum likelihood training through error exploration and dynamic oracles Nivre, 2012, 2013), and related techniques in imitation learning for structured prediction (Daumé III et al., 2009;Ross et al., 2011;He et al., 2012;Goodman et al., 2016).
our formulation is closely related to the framework of (Chang et al., 2015), where our oracle can be seen as an optimal roll-out policy which computes action costs without explicit roll-outs.