Highly local features have been used in recent state-of-the-art models Dozat and Manning, 2016;Shi et al., 2017).
dozat and Manning (2016) show that a locally trained arc-factored model can give the best reported accuracies on dependency parsing.