Although AsynGrad does the weight update in parallel from different threads, from the viewpoint of w w w it simply receives a sequence of gradient updates (but with potential errors, which is the major difference from non-parallel SGD learning).
the AsynGrad learning problem can be casted as a traditional SGD learning problem, and the only difference compared with traditional SGD is that the gradients are with errors.