Some previous work has been cautious in pointing out this ill-defined setting (Ravi and Knight, 2009;Christodoulopoulos et al., 2010), and we argue its inappropriateness for scenarios in which the test set is extremely small or even when no annotated reference corpus exists.
the problem of grounding the sequence of states or cluster IDs to POS tags without using any linguistic resource remains unsolved.