To determine the range in which word order affects model performance, we permute substrings in the context to observe their effect on dev loss compared to the unperturbed baseline.
we perturb the context as follows, where ⇢ 2 {shu✏e, reverse} and (s 1 , s 2 ] denotes the range of the substring to be permuted.