We define the Querys as The latent adaptation from β to α is presented as the crossmodal attention ( 1)Note that Y α has the same length as Q α (i.e., T α ), but is meanwhile represented in the feature space of V β .
the scaled (by √ d k ) softmax in Equation (1) computes a score matrix softmax (•) ∈ R Tα×T β , whose (i, j)-th entry measures the attention given by the i-th time step of modality α to the j-th time step of modality β.