The present paper presented a first step in this direction using a version of an existing TE dataset which was augmented with images that could be paired directly with the premises, since these were originally captions for those images.
it is important to note that in this dataset premise-hypotheses pairs were not generated directly with reference to the images themselves.