Formally, the cost function is defined as: where s 1 , s 2 are the input sentences, f the compositional layer (so f (s 1 ) and f (s 2 ) refer to sentence vectors), and y = 1 denotes a paraphrase relationship between the sentences; m stands for the margin, a hyper-parameter chosen in advance.
the cost function based on cosine similarity handles only directional differences, as follows: where is the cosine similarity of the two sentence vectors, w and b are the scaling and shifting parameters to be optimized, σ is the sigmoid function and y is the label.