4): Finally, as in the vanilla Transformer decoder, the third sub-layer is a feed-forward network that processes the representation for the next n+1 layer: The three sources of information (image, object labels, and web entity labels) are treated symmetrically in the above equations.
among the 3 sub-layers, we modify the encoderattention sub-layer by introducing a mechanism that combines information coming from different encoders.