Our MNMT model can be seen as an expansion of the attention-based NMT framework described in §2.1 with the addition of a visual component to incorporate spatial visual features.
we extract spatial image features for all images in our dataset using the 50-layer Residual network (ResNet-50) of .