In particular, we (i) leverage ultra-fine-grained semantic labels (e.g., "golden gate bridge" vs. "bridge") for featurization (Juan et al., 2019); and, (ii) focus on scenarios in which object detection modules trained on Visual Genome (VG) (Krishna et al., 2017) are applied to out-of-domain images: image captioning on the Conceptual Captions dataset (Sharma et al., 2018), and VQA on the VizWiz dataset (Gurari et al., 2018).
moreover, this 1.8% improvement is a weighted average across answer types; the per-answer-type numbers indicate that 8 evalai.cloudcv.org/web/challenges/challengepage/102/overview our approach achieves even better improvements on two of the more difficult answer types, "number" (+4.5%) and "rest" (+3.3%).