To avoid these, we propose to apply ensemble of the top-3 models for each policy.
we first generate distribution with the top-3 models independently with the same policy, and then take the arithmetic average of the three distributions as the final token distribution at that step.