We use here the popular method of ''diagnostic classifiers'' (Hupkes et al., 2018).
we treat the hidden activations produced by a CNLM whose weights were fixed after language model training as input features for a shallow (logistic) classifier of the property of interest (e.g., plural vs. singular).