Under these limitations, we show that different RNN variants have different computational power.
we show that the LSTM and the Elman-RNN with ReLU activation are strictly stronger than the RNN with a squashing activation and the GRU.