Therefore, a forgetting mechanism to "forget" less critical historical information, as is employed in LSTM (controlled by the forget gate f t ), becomes necessary.
while LSTM benefits from the flexible gating mechanism, its parametric nature brings optimization difficulties to cope with fairly long sequences, whose long range information dependencies could be better captured by identity connections.