A sentence contains rich syntactic information and can be modeled through sophisticated neural networks (e.g., convolutional neural networks (Kim, 2014;Kalchbrenner et al., 2014), recurrent neural networks (Sutskever et al., 2014;Le and Mikolov, 2014;Hill et al., 2016) and recursive neural networks (Socher et al., 2013)).
given a longer sentence, the effect of useful word vectors become smaller and thus the average vector is less reliable at capturing the semantics.