Despite the distinct training procedures, both algorithms combine the idea of fine-grained credit assignment and the entropy regularization, leading to positive empirical results.
it is not true because Q R (i.e., P Q R ) can condition on the correct reference y * .