RNMT+ is slower to train than the Transformer Big model on a per-GPU basis.
since the RNMT+ model is quite stable, we were able to offset the lower per-GPU throughput with higher concurrency by increasing the number of model replicas, and hence the overall time to convergence was not slowed down much.