This happens because the sensationalism scorer can make mistakes and RL can generate unnatural phrases which fools our sensationalism scorer.
how to effectively leverage RL with noisy rewards remains an open problem.