After obtaining a set of visually guided sentence features H qp = {h qp t } tp t=1 , we characterize the fine-grained matching behaviors between the visual and sentence features.
the matching behavior between the i-th visual and sentence features is defined as The instantiation of φ can be realized by different approaches, such as multi-layer perceptron (MLP), inner-product, or cosine similarity.