We apply blank noise variational smoothing (Kong et al., 2019) with a dropout rate of 0.25 to the tied embeddings to regularize the model.
for each document we sample a set of word types to drop and replace their word embeddings with the < UNK > token's word embedding.