(2017) which investigates a variant of it for continual learning (learning a new task while preserving performance on previously learned task) rather than domain adaptation.
our baseline fine-tuning models use the same hyperparameters, except that the learning rate is 4 times smaller and the validation frequency for early stopping 4 times higher.