The entity composite model predicts the next word based on modeling the conditional probability P (w|w, s(w), s(w), θ v ), which can be derived by where Ω(s(w)) is the set of words of the same type with w. To model the types of context word s(w) in P (w|w, s(w); θ v ), we consider learning a type embedding along with the word embedding by augmenting each word vector with a type vector when learning the underlying word representation.
we represent each word w as a vector of are the word vectors and type vectors learned by the model from the training corpus, respectively.