In a fully supervised setting where the correct semantic parse of each question is available, learning the policy function can be reduced to a sequence prediction problem.
in addition, we also disallow adding duplicates of actions that already exist in the state.