(2015) propose dataset splits with no lexical overlap between the train and test portions.
model's performance in a lexically-split setting is an overly pessimistic estimate of models' true performance -in a realistic scenario, the model will occasionally make predictions for pairs involving some of the concepts from the training set.