We found that time-constrained AMT annotators performed well (i.e., > 70%) accuracy for k ≤ 3 but struggled with examples involving longer stories, achieving 40-50% accuracy for k > 3.
trained annotators with unlimited time were able to solve 100% of the examples (Appendix 1.7), highlighting the fact that this task requires attention and involved reasoning, even for humans.