With the above criteria, we select all correct translation options as positive instances for the revising step, and randomly sample the same number of wrong translation options to be negative instances.
translation options that are used by the baseline system are included as negative instances.