Empirical results that are shown in Table 2 indicate that by only adding the auxiliary language modeling objective, we obtain better results on all downstream tasks.
a comparison of the P-LM + aux model and the P-LM model shows that the performance of SiATL on classification tasks is improved by the auxiliary objective.