In other words, the parallelization of SGD is not well suited to a large number of work- ers.
the parameter updates in AllVec are completely independent of each other, therefore AllVec does not have the update collision issue.