They are capable of modeling sequences of arbitrary length by repetitive application of a recurrent unit along the tokens in the sequence.
recurrent neural networks are known to have several disadvantages like the problem of vanishing and exploding gradients.