And instead of encoding these two channels individually, we design a cross connection between the APU-2 to learn the interactions both inside and between different modalities.
we use the updated query O f q to guide the middle attention layer of the video channel APU-2, in order to learn the query-aware video information v f q .