That implies taking an action based on the current observation, where the action is picking a wordỹ t from the vocabulary V. r(ỹ, y) is a reward function with r(ỹ, y) = 1 if y = y.
experiments on the benchmark datasets show that (1) imitation learning is constantly better than reinforcement learning; and (2) the pointer-generator models with imitation learning outperform the state-of-theart methods with a large margin.