In most of these operations, the examples in a minibatch can be processed in parallel.
in the sparse-dense products used when updating the parameters D and D , we found it was best to divide the vocabulary into blocks (16 per thread) and to process the blocks in parallel.