DIRECTNOISE Whereas BACKTRANS (NOISY) and BACKTRANS (SAMPLE) generate ungrammatical sentences with a reverse model, DIRECT-NOISE injects noise "directly" into grammatical sentences (Edunov et al., 2018;Zhao et al., 2019).
for each token in the given sentence, this method probabilistically chooses one of the following operations: (i) masking with a placeholder token mask , (ii) deletion, (iii) insertion of a random token, and (iv) keeping the original 3 .