Recent work focuses on learning the mapping from visual segments to the input text (Hendricks et al., 2017;Gao et al., 2017a;Liu et al., 2018;Hendricks et al., 2018;Zhang et al., 2018) and retrieving segments based on the alignment scores.
in order to successfully train a NLL model, a large number of diverse language descriptions are needed to describe different temporal segments of videos which incurs high human labeling cost.