As with BERT, we observe that the weights are only weakly concentrated for the relations and SPR tasks.
unlike BERT, we see only a weak concentration in the weights on the coreference task, which agrees with the finding of Tenney et al.