This implies that z ψ is trained to maximize the likelihood of different subsequent texts of different lengths, and thus is an approximately good starting point for any sequence.
a new state is computed during the training as follows: where r ∼ Bern(ρ) and ρ is a hyper-parameter for the probability of resetting a state.