Alternatively, future models might answer correctly only by picking up on spurious patterns, in which case a new development of the benchmark -using these models as adversaries -would place us in the same position as we are right now.
for instance, while ESIM+ELMo obtained 59% accuracy on SWAG, it obtains only 33.3% accuracy on HellaSwag.