As is well-known from other learning tasks, fine-tuning of the feature set can produce additional accuracy; we expect this to be the case in this task, too.
introducing features that capture contextual information is likely to improve the results.