The performance improvement is more pronounced when there is a mismatch between the LM and task domains, i.e., the non-Twitter domain tasks.
for the PsychExp and SCv2 datasets, sequentially unfreezing yields significant improvement in F 1 building upon our intuition.