Note that, an alternate way of collecting human judgments would have been to take the output of existing AQG systems and ask humans to assign answerability scores to these questions based on the presence/absence of the above mentioned relevant information.
when we asked human evaluators to analyze 200 questions generated by an existing AQG system, they reported that the quality was poor.