We conduct extensive experiments on five public datasets from various domains (summarized in Table 1 and detailed in Appendix A).
the cumulative reward from current time step to the end of an episode would cancel the intermediate rewards and thus reflect whether the current action improves the holistic label assignment or not.