The probability for the document V is given by: where F θ (V ) is the "free energy", which can be analytically integrated easily, and Z D is the "partition function" for normalization, only associated with the document length D. As the hidden state and document are conditionally independent, the conditional distributions are derived: where 3is the softmax units describing the multinomial distribution of the words, and Equation (4) serves as an efficient inference from words to semantic meanings, where we adopt the probabilities of each hidden unit "activated" as the topic features.
rSM is naturally learned by minimizing the negative log-likelihood function (ML) as follows: the gradient is intractable for the combinatorial normalization term Z D .