In order to abstract away from pos tagging errors, the algorithm is only evaluated on those words that were assigned the appropriate pos for a given word.
if the test set contains a singular noun, it is looked up in the automatically produced target language dictionary.