In addition, their method runs the NMT model for each retrieved sentence pair to obtain the NMT encoding and decoding information of the retrieved sentences as key-value memory to guide the NMT model for translating the real input sentence, which changes the NMT model structure and increases both the training-time and test-time computational cost.
at test time, running the NMT model for one retrieved sentence pair costs the same time as translating the retrieved source sentence with beam size 1.