To create the set of training instances, we run PathLSTM (Roth, 2016;Roth and Lapata, 2016) on all captions, and extract all semantic frame annotations whose roles are filled by a grounded entity.
sRL systems in NLP research use training data which have been carefully created by linguistic experts (e.g., Ruppenhofer et al.