We propose an Attention Gated Transformation (AGT) network, where each layer's feature generation is gated by a layer-specific attention mechanism (Bahdanau et al., 2014).
through distributing its attention to the original given text, each layer of the networks tends to in-crementally retrieve new words and phrases from the original text.