Motivated by this requirement and following the goal of developing unsupervised methods for counterfactual rewriting, we design a reconstruction objective for learning a noisy reproduction of the original ending.
we provide the model with both the original story and a masked context as input (S, [s], s 1 , [mask]) and train the model to reconstruct the original ending s 3:5 : where [s] denotes a separator token and [mask] is a special mask token.