This behaviour enable the model to retrieve information from the buffer to remember past words without relying solely on the RNN's internal "memory".
the model can maximize the features extracted about an input word, creating an advantage over other RNN-LMs that need to both extract features and keep context regarding the sequence in its connections.