Nevertheless, integrating these traditional gates inevitably increases computational overhead which is accumulated along token positions due to the sequen-tial nature of RNNs.
the weak parallelization of RNNs makes the benefits from improved model capacity expensive in terms of computational efficiency.