The self similarity of w in layer is (1) where cos denotes the cosine similarity.
the self-similarity of a word w in layer is the average cosine similarity between its contextualized representations across its n unique contexts.