Window representations for MemNNs have been shown to work well previously (Hill et al., 2016).
such structured memory is not employed in any existing attention-based neural network architecture for QA.