To tackle this issue, we further integrate cluster-level representations, at the cost of some further information loss, to create representations of lower dimension for inputs.
given all input documents with k cluster-level representations A ∈ R d×k for each document, we develop an autoencoder with one hidden layer that integrates these cluster-level representations to create a final representation for each document, vector a ∈ R d×m where m is the dimensionality reduction parameter and m × d is length of the representation (final layer of the encoder) and is smaller than d × k for m < k. Training a single-layer autoencoder corresponds to optimizing the learning parameters to minimize the overall loss between inputs and their reconstructions.