Because the prior frequency distribution for each person-question pair is collected from thousands of users with the assumption that most of them do not lie when answering questions in the Q20 game.
it is necessary to design a better reward function to estimate a non-zero immediate reward r t , and make the long-time return G t = T k=0 γ k r t+k+1 more informative.