To establish the potential of such models in word-based neural sequence-to-sequence systems, we integrate a web-scale count-based language model.
we use the modified Kneser-Ney 5-gram language model trained from Common Crawl (Buck et al., 2014), made available for download by Junczys-Dowmunt and Grundkiewicz (2016).