Therefore, we propose to rank all segments using early-summarized hidden representations as a mechanism for efficiently retrieving few top-ranked segments.
let h J denote the hidden states in the J-th block, where J < I.