We evaluate the generalization ability of PROVER compared to RuleTakers by training models on the train splits of DU0, DU1, DU2 and DU3, and testing the QA performance on the overall test set for DU5, which includes questions with higher depth than seen during training.
their task requires predicting the truth value of a statement by reasoning over a set of facts and rules, all expressed in natural language.