Our general strategy for learning relation vectors is based on averaging word vectors.
for each sentence s in which w i occurs before w j (within a distance of at most 10), we construct three vectors, based on the words a 1 , ..., a k which appear before w i , the words b 1 , ..., b l which appear in between w i and w j and the words c 1 , ..., c q which appear after w j : where we write v w for the vector representation of the word w. These vectors are then averaged over all sentences S ij where w i occurs before w j : Since we can similarly obtain such vectors from sentences where w j appears before w i , we end up with a relation vector whose dimensionality is six times higher than the dimensionality of the word embedding, which would be impractical in the kind of applications we envisage (see Section 4).