To evaluate the quality of the learned sense embeddings, we compute the similarity score between each word pair given their respective local contexts and compare with the human-judged score using Stanford's Contextual Word Similarities (SCWS) dataset (Huang et al., 2012).
given a list of word pairs with corresponding contexts, S = {(w i ,C t , w j ,C t )}, we calculate the Spearman's rank correlation ρ between human-judged similarity and model similarity estimations 4 .