Modern RNN architectures have been very successful in solving character-level NLP tasks (Chung et al., 2016b;Golub and He, 2016;Mikolov et al., 2012).
they do not make the learned linguistic structure explicit: rather it can be presumed to be cryptically encoded in the states of the hidden layers.