We train a log-linear model to predict the words that our subjects guess on training data, and we will check its success at this on test data.
from an engineering perspective, we do not actually need to predict the user's specific good or bad answers, but only whether they are good or bad.