To facilitate comparison with ShallowCNN, we matched our unsupervised embedding setting exactly with that of (Johnson and Zhang, 2016).
we trained the same four types of tvembeddings, which are embeddings of 5-and 9word regions, each of which represents the input regions by either 30K-dim bow or 200K-dim  bags of {1,2,3}-grams, retaining only the most frequent 30K words or 200K {1,2,3}-grams.