By using the predicted granularity levels and the context attentions in the Transformer, we are able to extract the phrasal and sentential paraphrase patterns from the model.
we extract the sentential templatesX  of X (orȲ of Y ) by substituting each fragment of words at the phrasal level by a unique placeholder such as $x.