In our experiments, we use the data 7 provided 7 http://nlp.cs.illinois.edu/ HockenmaierGroup/Papers/VL2016/ HodoshHockenmaier16_BinaryTasks_Data.tar by the authors for a subset of FLICKR-30K (Hodosh et al., 2013).
we consider four different types of distractions for the image descriptions, namely 1) Replace-Scene, 2) Replace-Person, 3) Share-Scene, and 4) Share-Person, which results 15548 correct and distracted caption pairs in total.