We consider three sets of word embeddings for our experiments: (i) word2vec 2 is trained on 100 billion tokens of Google News dataset; (ii) GloVe (Pennington et al., 2014) 3 is trained on aggregated global word-word co-occurrence statistics from Common Crawl (840B tokens); and (iii) syntactic word embedding trained on dependency-parsed corpora.
the model comprises k weight vectors w 1 , w 2 , ...w k , each of which is associated with an instantiation of a specific filter size.