Unlike SemEval-2007, the SemEval-2012 task is concerned exclusively with ranking substitutes; all the original participating systems were given the gold-standard substitutes and simply asked to put them in the correct order.
to score our own systems we use their own substitute lists, removing only those substitutes that do not also appear in the goldstandard list.