Another shortcoming of the RL-style methods is delayed punishment: For many NLP problems, including morphological string transduction, one can pinpoint actions that adversely affect the global score.
models did not learn much, which we attribute to sparse training signal as all action sequences producing the same y would incur the same sequence-level loss, including intuitively very wasteful ones, e.g.