In all cases, we see that the right choice of probe leads to considerably higher selectivity than dropout or no regularization.
for part-of-speech tagging, our chosen MLP-1 probe achieves 16.6 selectivity, up from 4.5, and on dependency head prediction, 6.2 selectivity, up from -0.7.