Transformer models have more advanced self-attention networks and more fine-grained multi-head attention mechanisms compared to RNN-based models with soft-attention.
transformer models have better performance in conventional translation tasks.