This is because the target word prediction task is dictionary-supervised in expanded setting rather than reference-supervised as in base setting.
our method still outperforms the simple pre-training method by a large margin.