This task also demands BERT to distinguish the characteristic of source and target domain for better knowledge transfer.
we propose BERT posttraining which takes BERT's pre-trained weights as the initialization for basic language understanding and adapts BERT by novel self-supervised pretrained tasks: domain-distinguish task and target domain masked language model.