Moreover, these results concern the expressive power of Transformers and do not apply to learning and generalization abilities.
transformers' ability to model formal languages requires further investigation.