f Θ is a composition function taking R w as input and outputting a single vector w as the word embedding of w. For the distributional "word-level" training, similar to prior work (Bojanowski et al., 2017), we adopt the standard skip-gram with negative sampling (SGNS) (Mikolov et al., 2013) with bag-ofwords contexts.
we note that other distributional models can also be used under the same framework.