Since the main difference between training and inference for sequence prediction tasks when predicting token y t is whether we use Figure 1: Scheduled Sampling the true previous token y t−1 or an estimateŷ t−1 coming from the model itself, they apply a sampling mechanism that will randomly decide, during training, whether to use the true previous token y t−1 or an estimateŷ t−1 coming from the model itself.
for every token, they flip a coin and decide whether to use the true previous token or an estimate coming from the model itself.