Our work is inspired by recent successes of memory-augmented neural networks on multiple NLP tasks Sukhbaatar et al., 2015;Miller et al., 2016;Gu et al., 2018), especially the efficient cache-like memory networks for language modeling (Grave et al., 2017;Daniluk et al., 2017).
the proposed approach augments NMT models with a continuous cache (CACHE), which stores recent hidden representations as history context.