When context information is not readily available, we use topic models to determine such context for individual inputs (Blei et al., 2003;Stevens et al., 2012).
we use Non-Negative Matrix Factorization (NMF) (Lin, 2007): Given a training set with n instances, i.e., X ∈ R v×n , where v is the size of a global vocabulary and the scalar k is the number of topics in the dataset, we learn the topic matrix D ∈ R v×k and context matrix C ∈ R k×n using the following sparse coding algorithm: where each column in C is a sparse representation of an input over all topics and will be used as global context information in our model.