Only the BERTScore metrics appear to positively correlate with human scores for counterfactual understanding, making them usable for evaluating generations across properties related to all three questions.
model Size and Pretraining Data We observe that models with more parameters are better at the counterfactual rewriting task than smaller models.