We use boldfaced w ∈ R k to denote a k-dimensional input word embedding, which can be pre-trained or derived from a character-level encoder (e.g., the character-level  PAR learns an orthogonal transformation M ∈ R k×k to reshape the input representation into a specific space, where the contextualized embedding vectors of a word in paraphrased contexts are collocated, while those in unrelated contexts are differentiated.
given two contexts S 1 and S 2 that both contain a shared word w, the contextual difference of a input representation w is defined by the L2 distance, Let P be the set of paraphrases on the training corpus, we minimize the following hinge loss (L H ).