Dropout (Srivastava et al., 2014) is also employed on formula (4) for regularization.
at training time, each element in g is randomly dropped out by multiplying a Bernoulli random variable with probability p of being 0.