Given a document D consisting of tokenized word sequence {w i : i = 1, 2, • • • , |D|}, its perplexity L(D|M j ) with respect to a LM M j is computed as: where |D| is the number of words in D and h i are the history words for w i , and P (w i |h i ; M j ) is the probability M j assigns to w i , when it follows the history words h i .
we used decision tree (reduced error pruning (Quinlan, 1987)) regression, decision tree regression with bagging (Breiman, 1996), support vector regression (Smola and Scholkopf, 1998) using polynomial kernel of degree two, 2 linear regression and Gaussian process regression (Rasmussen and williams, 2006).