In literature, there is a large bundle of studies for injecting some level of word order information into word embeddings (Qiu et al., 2014;Johnson and Zhang, 2015;Lai et al., 2015;Ling et al., 2015;Trask et al., 2015) or into general word-space models (Sahlgren et al., 2008;De Vine and Bruza, 2010;Basile et al., 2011).
see figure 3 for a schematic view of the order preserving model.