The dataset contains 74 restaurant reviews typed with phrase suggestions.
we can obtain an unbiased estimate of the reward that h ✓ would incur using importance sampling:R the variance of this estimate can be unbounded because the importance weights h ✓ (y i |x i )/p i can be arbitrarily large for small p i .