Here, for an n-layer MLP encoder on word counts c i , we build on that idea, and use as representations a weighted sum over θ i and the internal states of the MLP, h i , with weights to be learned by the downstream classifier.
5 for any sequence-to-vector encoder, f s2v (x), we propose to augment the vector representations for each document by concatenating them with a weighted combination of the internal states of our variational encoder (Peters et al., 2018a).