Besides achieving extra long context and resolving fragmentation, another benefit that comes with the recurrence scheme is significantly faster evaluation.
during evaluation, the representations from the previous segments can be reused instead of being computed from scratch as in the case of the vanilla model.