We found that in the verification of answer-options, our annotations were in high agreement (98%) with those obtained from mechanical turk.
that was not the case for the verification of multi-sentence questions.