While the former gate allows the model to forget the previous states, the latter gate decides how much the model updates its content.
our word-level decoder is released from the pressure of memorizing the long (interchunk) dependencies and can focus on learning the short (intra-chunk) dependencies.