One of the weaknesses of these approaches, however, is that they do not take word ordering into account during the learning process.
word-based approaches based on RNNs that consider sequence information have been presented, but they are not competitive in terms of speed or quality of the embeddings (Mikolov et al., 2010;Mikolov and Zweig, 2012;Mesnil et al., 2013).