Moreover, it is delightful that our work can achieve a rather good result with a relatively simpler model architecture.
the information provided by standard pretraining with unlabeled data is more general, while that of heterogenous resource is more relevant to our task, hence is more informative and evident.