We also observed that adding visual features to textual features improves performance in some cases: multimodal features perform better than textual features alone both for object labels (CNN+O) and for image descriptions (CNN+C).
adding CNN features to textual features based on object labels and descriptions together (CNN+O+C) resulted in a small decrease in performance.