The second was a TensorFlow-based RNN with an attention mechanism (Mnih et al., 2014), using an overall architecture similar to that used in a system for end-to-end speech recognition (Chan et al., 2016).
we used a 4-layer pyramidal bidirectional LSTM reader that reads input characters, a layer of 256 attentional units, and a 2-layer decoder that produces word sequences.