We do this by integrating the partial environment model into the policy function.
we modify the log-linear policy p(a|s; q, θ) by adding lookahead features φ(s, a, q) which complement the local features used in the previous model.