Other researchers have also shown that language models can "falsely" solve the task.
they might be taking advantage of dataset failures or artifacts on the input sentences in order to guess the answer (Gururangan et al., 2018; Agrawal et al., 2016; Levy et al., 2015).