Based on the post-trained BERT, we now could utilize the adversarial training to abandon the distilled domain-specific features to derive the domaininvariant features.
a sentiment classifier and a domain discriminator is designed operating on the hidden state h [CLS] of the special classification embedding [CLS].