It would be interesting to see how such states capture the meaning of questions.
the encoders pre-trained in this manner are subsequently fine-tuned according to the discriminative criterion described already in Section 3.