This problem has been addressed in the context of robot direction following (Kollar et al., 2010;Tellex et al., 2011) and visual attribute learning (Matuszek et al., 2012).
this work is less semantically expressive than LSP and trained using more supervision.