ON-LSTM is better at modeling hierarchical structure by introducing a syntax-oriented inductive bias, which enables RNNs to perform tree-like composition by controlling the update frequency of neurons.
we stack SANs encoder on top of ON-LSTM encoder (cascaded encoder).