Once the dataset is created, we formally define a new task, visual pronoun coreference resolution (Visual PCR), and design a novel visual-aware PCR model VisCoref, which can effectively extract information from images and leverage them to help better resolve pronouns in dialogues.
we align mentions in the dialogue with objects in the image and then jointly use the contextual and visual information for the final prediction.