We also apply CVT on the first hidden layer of the sentence learner to further improve the performances of auxiliary tasks, utilizing the views introduced in  for sequence tagging and dependency parsing.
we add the following four auxiliary prediction modules to the model: The "forward" module makes predictions without seeing the right context of the current word.