RNNs have largely replaced approaches such as the fixed-window-size feed-forward networks of Durrett and Klein (2015) in part due to their ability to capture global context.
rNNs are not the only architecture capable of summarizing large global contexts: recent work by Vaswani et al.