However, the bilinear probe achieves 16.7 selectivity, compared to −0.7 by MLP-1 and 1.3 by MLP-2.
with no regularization, modest gains in linguistic task accuracy through MLP probes over linear/bilinear probes are tempered by losses in selectivity.