majority of works are trained via supervised learning that requires a large corpus of caption labeled images such as the MS COCO caption dataset [Lin et al., 2014].
the dataset is constructed with 1.2M images that were asked the annotators to provide five grammatically plausible sentences for each image.