They use these vectors as input to a logistic regression classifier and achieve state-of-the-art performance on sentiment classification of movie reviews.
they did not consider the effect of this model modification directly on the task of language modelling.