To avoid such cases, we propose to evaluate a candidate's fitness by comparing the sentence's contextualized representation before and after the substitution for validation.
for a word x i , we use the concatenation of its representations in top four layers in BERT as its contextualized representation.