L RAML,θ can be trained in a MLE-like fashion but sampling from the distribution Q φ (X) is intractable in the adversarial setting, because Q φ (X) varies with the discriminator D φ .
we introduce importance sampling to separate sampling process from D φ and obtain the final loss function: where P s (X) denotes a stationary distribution and To optimize this loss function, we first construct the fixed distribution P s (X) to get samples, and devise the proper reward function r φ (X) to train the generator in a stable and effective way.