The RNN language model decomposes into per-word probabilities via the chain rule.
in order to allow for more expressivity over long range context we do not require the discriminative model scores to factorize over the elements of y, addressing a key limitation of RNNs.