Recently the connection between less formulaic language and simple actions has been explored successfully in the context of simulated worlds (Branavan et al., 2009;Goldwasser and Roth, 2011;Branavan et al., 2011;Artzi and Zettlemoyer, 2013;Andreas and Klein, 2015) and videos (Malmaud et al., 2015;Venugopalan et al., 2015).
to our knowledge, there is no body of work that focuses on understanding the relation between natural language and complex actions and goals or on explaining flexibly the actions taken by a robot in natural language utterances.