We used Amazon Mechanical Turk to conduct human evaluation.
randomly selected 100 documents and their four summaries generated by DIS w/ PAR, Lead-3, no-attn, and SummaRuNNer were shown to the workers.