In order to inject the global semantics, we perform an element-wise addition of h ctxt to the mid-level feature along the time axis, and feed them into the bi-directional LSTM one at a time, producing a hidden state ← → h a2 t ∈ R D 2 at each step: In many captioning models Yu et al., 2017;Laokulrat et al., 2018;Long et al., 2018), semantic attention has been independently used from temporal/spatial attention.
it can be troublesome because there may exist some discrepancies between the two attentions i.e.