Indeed, Appendix A in (Chen et al., 2017) provides concrete IRNNs for recognizing the languages a n b n and a n b n c n .
the SRNN update equation is: By properly setting U and W, one can get certain dimensions of h to update according to the value of x, by h this counting behavior is within a tanh activation.