We consider a toy classification task: determining whether a conversation is taken by a new coun-selor who has experienced less than 20 conversations (S 0 ), versus a tenured counselor who has taken between 100 and 120 (S), on the basis of the language in the counselor's messages.
we formulate a paired prediction problem of distinguishing between two conversations taken by the same counselor at S 0 and S. Over a random subset of 10% of counselors in our data-comprising 3,075 conversation pairs-we train a logistic regression model using bigrams in counselor messages as features; we perform 10fold cross-validation and ensure that no counselor spans multiple folds.