Recently, Yannakoudakis and Briscoe (2012) systematically analyzed a variety of coherence modeling methods within the framework of an automated assessment system for non-native free text responses and indicated that features based on Incremental Semantic Analysis (ISA), local histograms of words, the part-ofspeech IBM model, and word length were the most effective.
for coherence modeling, we again use the J48 decision tree from the Weka machine learning toolkit (Hall et al., 2009) and run 4-fold crossvalidation on the 600 annotated responses.