To address this issue, we propose the AdaptaBERT model for unsupervised domain adaptation, which adds an additional fine-tuning objective: masked language modeling in the target domain.
we apply a simple two-step approach: 1.