Based on a limited context of size n words, the surprisal value of the following word w n+1 corresponds to the negative log-probability: There are two properties inherent to surprisal: (1) units with low probability convey more information than those with high probability, and (2) information conveyed by a unit is crucially dependent on its context.
for POS tagging and syntactic parsing, we use the Stanford NLP toolkit (Manning et al., 2014).