This strategy is helpful in practice to use richer representations for some NLP problems.
such a clean modular separation of representations computed with and without attention is harder to realize in attention-based RNNs.