As already mentioned, this result can probably be improved by using a subset of readers or by weighting training examples, e.g., by importance weighting (Shimodaira, 2000).
they use use word lengths and frequencies, like us, as well as distances between tokens (important in a transitionbased model), and, finally, the history of previous decisions.