Our method employs topic distribution of a linguistic unit as its contextual information.
given a linguistic unit S = [w 1 , w 2 , ..., w n ], its topic distribution C = {p(topic 1 |S), p(topic 2 |S), ...} and the topic-specific embeddings of all words, we construct the representation of S as follows: 1) We calculate the topic-biased representation of S under topic t by composing the topic-specific word embeddings as follows: where f is a base composition model, which composes a sequence of vectors into a single representation.