As an alternative, we present and evaluate new sequencepreserving ways of attention.
attention can be thought of as weighted selection: some input elements are highly weighted, others receive weights close to zero and are thereby effectively not selected.