In this case, since the out-ofdomain labeled dataset (MRDA) is much larger, it overwhelms the model inducing features that are not relevant for the task in the target domain.
when we provide the models with some labeled in-domain examples in the semisupervised (50%) setting, we observe about 11% absolute gains in QC3 and BC3 over the corresponding Merge baselines, and 7 -8% gains over the corresponding Fine-tune baselines.