We further show that a diagnostic classifier trained on the biased baseline neural network also does not generalize to new samples.
the biases detected in Elazar and Goldberg (2018) seem restricted to their particular data sample, and would therefore not bias the decisions of the model on new samples, whether in-domain or out-of-domain.