To quantify this effect and compare to QRPE, we pair each irrelevant image-question pair (I − , Q) from VTFQ with a relevant image from the VQA dataset.
we find the nearest neighbor question Q nn in the VQA dataset to Q based on an average of the word2vec (Mikolov et al., 2013) embedding of each word, and select the image on which Q nn was asked as I + to form (I + , Q, P, I − ) tuples like in our proposed dataset.