Because of the discrete choice of neutral words, the loss is no longer differentiable over the neutralization module.
we formulate it as a reinforcement learning problem and use policy gradient to train the neutralization module.