The learning signal weights the gradient for each latent sample z.
in REiN-FORCE, to cope with the high variance of the learning signal, it is common to use a baseline b(x) to stabilize learning, and re-define the learning signal as in STRUCTVAE, we define where log p(x) is a pre-trained LSTM language model.