For instance, the Pointer Networks (Vinyals et al., 2015) uses attention over input timesteps.
it has no power to rewrite information for later use, since they have no "memory" except for the RNN states.