Pairs of utterances which are widely separated in the discourse are unlikely to be directly related-even if they are part of the same conversation, the link between them is probably a long chain of intervening utterances.
if we run our classifier on a pair of very distant utterances, we expect it to default to the majority class, which in this case will be different, and this will damage our performance in case the two are really part of the same conversation.