However, the Bernouli distribution is hard to reparameterize, where we need to enumerate 2 V different values to compute the expectation over the stochastic dropout vector b.
we follow Wang and Manning (2013) to use a continuous Gaussian approximation, where the Bernouli noise b is replaced by a Gaussian noise z: where It is worth noting that α and p are one-to-one corresponded, and α is a monotonously increasing function of p. For more details, please refer to Wang and Manning (2013).