Training an NLI model in this end-to-end manner assumes that any inference type involved in the sentence-level decision may be learned from the training data.
recent work created challenge datasets which show that these modelswhen trained on the original NLI datasets-fail when they need to make inferences pertaining to certain linguistic phenomena, often ones which are not sufficiently represented in the training data.