Balduzzi and Ghifary (2016) applies type-preserving transformations to discuss the capacity of various simplified RNN architectures.
typically, c t 1 is multiplied with a parameter matrix to compute f t , e.g., the inclusion of V f c t 1 makes it difficult to parallelize the state computation: each dimension of c t and f t depends on all entries of c t 1 , and the computation has to wait until c t 1 is fully computed.