A constant number of arithmetic operations suffice to compute the final expected loss value and its gradient.
computing the gradient does not increase the algorithmic complexity when compared to conventional approaches using coordinate ascent and random directions.