The nonconsecutive feature interaction is well addressed by the memory cell and different gates in LSTM unit.
the forget gate is able to decay the information according to the context rather than a fixed decaying weight in tensor product based CNNs.