To conclude this section, we believe the formalization of the syntax and semantics of graphical representations in a form compatible with the syntax and semantics of natural language, as in the scheme in Figure 3, may be a point of departure for investigating how the graphical or visual context helps to resolve natural language ambiguities at different levels of representation and processing.
if the expression is supported by an overt gesture indicating the city of Paris, for instance, it would be deictic and its interpretation would have to be worked out with the same machinery; although in this latter situation the translation relation between graphical and linguistic referents could be asserted directly in the translation model as the gesture would render unnecessary the constraint satisfaction part of the deictic inference.