Although there is a fair bit of research on generating naturallanguage descriptions of images (Feng and Lapata, 2013;Yang et al., 2011;Ordonez et al., 2011), there is significantly less work on describing videos (Barbu et al., 2012;Das et al., 2013;Rohrbach et al., 2013;Senina et al., 2014).
much of the research on videos utilizes artificially constructed videos with prescribed sets of objects and actions (Barbu et al., 2012;Yu and Siskind, 2013).