In this sense, at each time step new information is processed and previous information still exists but is 'diluted' in the hidden state (due to the forget gate).
as some local interactions that are more informative, e.g.