These models differ from more traditional recurrent neural networks in different aspects.
while recurrent models can process sequences (in NLP, typically word embeddings) in order, transformers use a joint model of the right and left context of each word in order to encode an entire sequence or document.