The Back-Propagation Through Time (BPTT) variant of SGD enables to sum-up gradients from consecutive time steps before updating the parameters of the RNN and to handle the long-range temporal dependencies in the hidden s and output y sequences.
each word w i is assigned to a class c i and the corresponding log-likelihood is effectively log where j is the index of word w i among words belonging to class c i .