Thus, for 50% of the data, the model has access only to the QA conf and QA f score reward signals, since the ROUGE-L reward can only be computed on supervised batches.
for all the unsupervised setups, in order to keep consistency in the reward signal throughout the training, we multiply by a factor of 2 the weight associated with ROUGE-L when this reward is computable, and set it to 0 otherwise.