Joint Learning: To extend conventional embedding learning, we combine context learning from an original corpus with external knowledge encoded by a regularizer, where the shared vocabulary set forms a bridge connecting the two spaces.
the objective function for CBOW with integrating the regularizer can be formulated as maximizing where not only w i , but also R(w i ) is predicted by the context words w i+j via their embeddings υ i+j .