We train the generation models using SGD and truncated BPTT (Elman, 1990;Mikolov et al., 2010), as in language modeling.
we split each y 1:T into contiguous blocks of length 100, and backprop both the gradients with respect to the current block as well as with respect to the encoder parameters for each block.