In contrast, AWD-LSTM-DOC constantly decreased the perplexity and achieved a lower value than the other methods with ordinary SGD.
we conclude that DOC positively affects the training of language modeling.