We call models trained on datasets augmented with unlabeled corpus data or random strings DA-U or DA-R, respectively.
we thus conclude that multi-task training (instead of simple data augmentation) is crucial for the use of unlabeled data.