Kobayashi (2018) showed that replacing words with other words that were predicted from the sentence context using a bi-directional language model yielded a 0.5% gain on five datasets.
training a variational auto-encoder or bidirectional LSTM language model is a lot of work.