Similarly to LSTMs, GRU networks introduce a mechanism that facilitates the retention of information over multiple time steps.
the GRU computes the hidden state at current time step h t , as the linear combination of previous activation h t−1 , and a new candidate activationh t : where is elementwise multiplication, and the update gate activation z t determines the amount of new information mixed in the current state: The candidate activation is computed as: The reset gate r t determines how much of the current input x t is mixed in the previous state h t−1 to form the candidate activation: where W, U, W z , U z , W r and U r are learnable parameters.