Globally, except for the NUR task, the metaembedding configurations using pre-trained models are slightly better than the ones using indomain models and enhance the performance of the SentEmb model.
they outperform the SentEmb baseline and are ranked among the three best models for the SemEval tasks.