This difference is important because in the wikitext2 dataset the sentences are not shuffled and, therefore, are sequentially related to each other.
systems that carry sequential information from previous sentences into the current sentence have an advantage in that they utilise contextual cues from the preceding sentence to inform the predictions at the start of the new sentence.