We also conduct experiments on the cross-lingual classification task  using the cross-lingual language inference (XNLI) dataset .
we add a linear classification layer on top of the first hidden state of our pre-trained model and fine-tune its parameters on the English NLI dataset.