To verify the importance of explicitly modeling the entropy from future steps, we compared ERAC with the variant that only applies the entropy regularization to the actor but not to the critic.
the τ is set to 0 when performing policy evaluating according to Eqn.