Second, using the diversity regularization in Figure 5(b) can encourage each layer to contribute more equally to the aggregation.
for the En⇒De task, we trained on the widely-used WMT14 dataset consisting of about 4.56 million sentence pairs.