(2015), we propose to apply dropout to the cell update vector g t as follows: Different from methods of (Moon et al., 2015;Gal, 2015), our approach does not require sampling of the dropout masks once for every training sequence.
as we will show in Section 4, networks trained with a dropout mask sampled per-step achieve results that are at least as good and often better than per-sequence sampling.