At each state s * i on the solution path of a training example (x, y * ), we create an example by labeling optimal action a * i ∈ A(s * i ) as the only relevant action, and then try to learn a ranking function that can rank actions such that the relevant action a * i is in the top b actions, where b is the input pruning paramter.
we have a rank learning problem, where the learner's goal is to optimize the Precision at Topb.