We denote it g(s) for simplicity, where g is a multi-layer perceptron with input s = F t (S; θ t ) and parameters θ i Cluster information (C g ) The cluster information leverages the fact that two sentences describe, or not, the same underlying reality.
the goal is to measure if two sentences are visually equivalent (assumption (1) in Section 3.1) without considering the content of related images.