To avoid such cases, we propose to evaluate a candidate's fitness by comparing the sentence's contextualized representation before and after the substitution for validation.
bERT (Keep/Mask) are the baselines that uses bERT unmasking/masking the target word to propose candidates and rank by the proposal scores.