Whereas previous studies in generation mainly used human evaluation to compare different systems, or to correlate human and automatic evaluations, our primary interest is the agreement or correlation between human rankings.
we explore the hypothesis that this agreement is higher in certain contexts than in others.