In this work, we demonstrate that certain attention heads of a visually grounded language model actively ground elements of language to image regions.
some heads can map entities to image regions, performing the task known as entity grounding.