Hidden layers of RNNs can preserve the historical sequential information, and can be used as the representations of the texts (Dai and Le, 2015).
both CNNs and RNNs are essentially 'flat' models, where structural information (e.g.