On short text, we also found that attention (or the gated pooling mechanism from GRNN) did not really help make any significant improvements over the vanilla LSTM model and a qualitative explanation to why this is so is deferred to the next section.
attention helps for long text (such as debates), resulting in Attention LSTMs becoming the strongest baseline on the Debates datasets.