In this paper, we remove this limitation by proposing a novel encoder-decoder architecture with a structured fixed-size representation of the input that still allows the decoder to explicitly focus on different parts of the input.
our NMT system has both the capacity to attend to various parts of the input and to produce static representations of input sentences.