While semantically invariant text transformations can remarkably alter a model's predictions, the complementary problem of model undersensitivity is equally troublesome: a model's text input can often be drastically changed in meaning while retaining the original prediction.
previous works (Feng et al., 2018;Ribeiro et al., 2018a;Welbl et al., 2020) show that even after deletion of all but a small fraction of input words, models often produce the same output.