The estimated future rewards is then incorporated in the small-width beam search to simulate the performance of a large-width beam search.
given a K l -width beam search we want to simulate, at time step t it generates a set of partial responses For one partial response y t , its future reward with regard to beam width of if a partial response presents in the top-K responses from the given K l -width beam search (K < K l ) at future time steps, we assign it with a positive future reward.