The 100-billion-token corpus used in the original publication is not publicly available.
for the training of the LSTM models, we used the English Gigaword Fifth Edition (Linguistic Data Consortium (LDC) catalog number LDC2011T07).