In this work, we use a targeted evaluation approach (Marvin and Linzen, 2018) to elicit evidence for syntactic state representations from language models.
we examine language model behavior on artificially constructed sentences designed to expose behavior that is crucially dependent on syntactic state representations.