Therefore it seems reasonable that the hidden layer is not only related to the activity when the word is on the screen, but also related to the activity before the word is presented, which is the time when the brain is integrating the previous words to build that context.
as the word i and subsequent words are integrated, the context starts diverging from the context of word i (computed before seeing word i).