In other words, its hidden layers are required to memorize the long-term dependencies and orders in the target language.
in our word-level decoder, the hidden state iterates only over the length of a chunk and then generates an end-of-chunk token.