We also introduce a simple yet effective strategy to incorporate visual information into any neural network beyond weights from pretrained networks.
we consider the tags identified in an image as an additional textual input, and leverage pretrained word embeddings as usually done with regular text.