Although they tried to fix this problem by feeding randomly-permuted comments to the model, a different permutation of the same set of comments leads to a different Q-value prediction.
the agent may not consistently select the predicted optimal action.