We train head-modifier embeddings for each token, and use such embeddings as pre-trained embeddings.
we use an approach similar to training word2vec embeddings but focusing on head modifier relationships instead of co-occurrence relationships.