The word drop probability is normally set to 0.25, since using a higher probability may degrade the model performance (Bowman et al., 2016).
we observe that these tricks do not solve the degeneracy for the VHRED in conversation modeling.