In this work, we focus on jointly leveraging the contextual and visual information to resolve pronouns.
we formally define the visual-aware pronoun coreference resolution as follows: Given an image I, a dialogue record D which discusses the content in I, and an external mention set M, for any pronoun p that appears in D, the goal is to optimize the following objective: where F (•) is the overall scoring function of p refers to c given I and D. c and s denote the correct mention and the candidate mention, and C and S denote the correct mention set and the candidate mention set, respectively.