We introduce the notion of recurrence into our deep self-attention network.
instead of computing the hidden states from scratch for each new segment, we reuse the hidden states obtained in previous segments.