Perplexity: The perplexity of the n-gram model on the Wikipedia test data was 157.5, much higher than the perplexity of the single-task RNN (78.65) and the multi-task RNN (61.10).
perplexity tracked accuracy on our syntactic data set -an unsatisfying outcome given our goal of dissociating perplexity and our syntactic evaluation method, but an expected one given that each model was conditioned on richer information than the previous one.