Note, however, that peer-review helpfulness is rated for the whole review, which can include multiple idea units.
2 in our study, we calculate the distribution of feedbackType values (praise, problem, and summary) (kappa = .92), the percentage of problems that have problem localization -the presence of information indicating where the problem is localized in the related paper-(kappa = .69), and the percentage of problems that have a solutionthe presence of a solution addressing the problem mentioned in the review-(kappa = .79) to model peer-review helpfulness.