We worked on the Flickr30k entities dataset, where noun phrases are given and VGP supervision is available, extracting VGP in an end-to-end manner without supervision in other datasets such as the Microsoft COCO caption dataset (Chen et al., 2015) is a more realistic scenario and could be more interesting.
figure 4a shows such an example, where the SNN (fV+CCA) model fails to identify these two entities "a group of order men" and "a group of people" as VGPs due to the diverse textual descriptions of the the same visual concept.