1 Since the task's introduction, there has been no formal consensus around which of the two approaches offers the better cost-benefit tradeoff: precise (at risk of being impractical), or organic (at risk of being ill-defined).
the prevailing assumption across annotation methods is that there is a single ''true'' inference about h given p that we should train models to predict, and that this label can be approximated by aggregating multiple (possibly noisy) human ratings as is typical in many other labelling tasks (Snow et al., 2008;Callison-Burch and Dredze, 2010).