Multi-head attention with diversity: We employ K-head attention networks to attend to the visual objects in an image as well as the textual semantics in a sentence then generate fixedlength image/sentence representations for alignment.
the k-th attended German sentence representation g k is computed by: where a k i is the attention weight, W k g ∈ R H×Hattn , W k cg ∈ R Hc×Hattn is the learnable transformation matrix for German.