One is that these type of sentences can get a high generation probability since the generator is actually a language model.
considering the simplicity of the model and the ease of training, we adopt the neural constrained language model of  as the generator.