Therefore, the attention weight between the n-th word h s n and the t-th video frame h v t is determined by not only the content of the video and sentence but also their interaction status.
we design one network to compute the relevance score of one video frame with respect to each word: where vector w, matrices W * , bias vector b, and bias c are the network parameters to be learned.