A common practice is to apply RL on a neural sequence-to-sequence (seq2seq) framework with the action space being the output vocabulary in the decoder.
it is difficult to design a reward function that can achieve a balance between learning an effective policy and generating a natural dialog response.