We aim to provide a dataset that merges the best qualities from all of the previous datasets as well as focus on multimodal compositionality.
we collect a new large-scale dataset that is built on natural video content with rich dynamics and realistic social interactions, where questionanswer pairs are written by people observing both videos and their accompanying dialogues, encouraging the questions to require both vision and language understanding to answer.