Prior work on gender debiasing (Bolukbasi et al., 2016;Zhao et al., 2018b) showed that the vector difference between the embeddings for malefemale word-pairs such as he and she accurately represents the gender direction.
such a re-annotation exercise would be costly and inconsistent with the original ratings.