The VQA-X dataset contains 2, 000 question-image pairs from the VQA v2 validation set with human annotations indicating the objects which most influence the answer to the question.
we used Earth Mover Distance (EMD) (Rubner et al., 2000) to compare the highly-attended objects in the VQA process to the objects highlighted by human judges.