Of course, we do not have enough s-t parallel data to obtain a good estimate of the true backtranslation distribution P s (X|y) (otherwise, we can simply use that data to learn θ).
we posit that even a small amount of data is sufficient to construct an adequate data selection policy Q(X|y) to sample the sentences x from multilingual data for training.