To enable human-robot communication and collaboration, recent years have seen an increasing amount of works which aim to learn semantics of language that are grounded to agents' perception (Gorniak and Roy, 2007;Tellex et al., 2014;Kim and Mooney, 2012;Matuszek et al., 2012a;Liu et al., 2014;Liu and Chai, 2015;Thomason et al., 2015Thomason et al., , 2016 and action (Matuszek et al., 2012b;Artzi and Zettlemoyer, 2013;Misra et al., 2014Misra et al., , 2015She and Chai, 2016).
for verb semantics, recent works explored the connection between verbs and action planning Misra et al., 2014Misra et al., , 2015She and Chai, 2016), for example, by representing grounded verbs semantics as the desired goal state of the physical world that is a result of the corresponding actions.