This connection allows SRNN to use at the current time step hidden layer's states of previous time steps.
the hidden layer of SRNN represents all previous history and not just n − 1 previous inputs, thus the model can theoretically represent long context.