Motivated by recent studies (Clark et al., 2019;Lin et al., 2019; which posit that individual attention heads can encode linguistic information, we analyze attention patterns in the self-attention heads of our models.
we search for heads that can individually perform coreference resolution.