Our training scheme builds on their work, with the following differences: we use separate model parameters for source and target texts (encoder and decoder), and we also use a different masking scheme.
we mask a varying percentage of tokens, only from the target, and do not replace input tokens with noise.