In addition to marking only a subset of the incorrect tokens at inference time, we also train new models for which the training data also only had a subset of incorrect tokens marked.
we train three models QE25, QE50, QE100 for which either 25%, 50% or 100% of incorrect guess tokens were marked.