The motivation comes from the fact that self-attention is superior in capturing long distant dependency, as shown in (Vaswani et al., 2017).
we first use a word-level LSTM encoder to obtain the fixed-dimensional representation of each context.