The methods proposed in (Gao et al., 2017;Hendricks et al., 2017) learn a common embedding space shared by video segment features and sentence representations, in which their similarities are measured.
moment context network (MCN) (Hendricks et al., 2017) learns a shared embedding for video clip-level features and language features.