The dropout has been applied in each hidden layer and in the final classification layer.
notice that other low-dimensional approximations of kernel functions have been studied, as for example the randomized feature mappings proposed in Rahimi and Recht (2008).