The application of a curriculum is based on the often unspoken assumption that the representation of a complex pattern can be reached more easily from a simpler pattern.
we find that effectively representing shorter scaffolds actually makes a language model less effective at generalizing a long-range rule, as found by Zhang et al. (2018).