With the above formula, the agent is optimized to the minimization of the model divergence between its own model and the ensemble model.
integrating the above regularization term into the training objective straightforwardly is problematic in practice.