In word-level language modeling, the input to the network is a sequence of words, and the network is trained to predict the next word in the sequence with a softmax classifier.
for a length-T sequence, denote y t = x t+1 for t = 1, .