Thus, the cache is, in a sense, complementary to the standard model, since it especially helps regenerate words from the long-range context where the latter falls short.
the cache also hurts about 36% of the words in PTB and 20% in Wiki, which are words that cannot be copied from context (C none ), as illustrated by bars for "none" in Figure 7.