This heuristic is proposed in hopes of better training the variational latent space at the beginning stages.
experiments show that such simple heuristic does not help much, and is worse than the principled variational attention mechanism in terms of all BLEU and diversity metrics.