As described in §2.2, our model is evaluated by the loss suffered at every timestep, where the loss is defined as the negative log likelihood of the model on text at timestep w t .
at each timestep t, we need to predict (the distribution of) w t .