In order to evaluate the system, we record whether the calculated entropies match the order of each hypernym-hyponym pair.
we count a pair as correctly represented by our system if w 1 is a hypernym of w 2 and KL(w 1 ) < KL(w 2 ) (or, in the case of the baseline, SI(w 1 ) < SI(w 2 ) where SI is selfinformation).