All heads in pr j are further concatenated as a mixed embedding m j ∈ R d model in turn.
rui Wang was partially supported by JSPS grantin-aid for early-career scientists (19K20354): "Unsupervised Neural Machine Translation in Universal Scenarios" and NICT tenure-track researcher startup fund "Toward Intelligent Machine Translation".