The model, φ, is trained on one source language, and the policy is learned on a different source language.
policy learning uses Alg 1, with the small change that in step 3 the model is initialised using φ. the learned policy can exploit the knowledge from cross-lingual initialisation, such that it can figure out which aspects that need to be corrected using target annotated data.