This improved agreement prediction accuracy, and the effect persisted over the course of the sentence, indicating that this information has an effect on the model.
they did not report the effect on overall model quality, for example by measuring perplexity.