We found that time-constrained AMT annotators performed well (i.e., > 70%) accuracy for k ≤ 3 but struggled with examples involving longer stories, achieving 40-50% accuracy for k > 3.
overall, we find that the GAT baseline outperforms the unstructured text-based models across most testing scenarios (Table 2), which showcases the benefit of a structured feature space for robust reasoning.