For each group of RL models, the column of the optimization objective is highlighted.
we show that reinforcement learning on coarse dialogue acts avoids degenerate solutions, which was a problem in Li et al.