Johnston (1998aJohnston ( , 1998b utilized techniques from natural language processing (unification-based grammars and chart parsers) to extend the unification-based approach and enable handling of inputs with more than one gesture, visual parsing, and more flexible and declarative encoding of temporal and spatial constraints.
to the unification-based approaches, which separate speech parsing and multimodal integration into separate processing stages, Bangalore (2000, 2005) proposed a one-stage approach to multimodal understanding in which a single grammar specified the integration and understanding of multimodal language.