Lawrence and Riezler (2018) and Berant et al.
we train s t+1 from all the examples generated in iterations 0 • • • t. in every iteration more and more examples are labeled, and a better scoring function is trained.