In R (www.r-project.org) notation, we can express this model as a=0.2 # acceptance rate s=400 # submissions r=3 # reviews per paper g=0.7 # mixture of good to random reviews gold = rbinom(s, 1, a) good = matrix(rbinom(s*r, 1, g), ncol=r) score = good*gold + (1-good)*matrix(rbinom(s*r,1,a),ncol=r) accept = rank(jitter(apply(score,1,sum)))>(s*(1-a)) precision = sum(accept * gold)/sum(accept) recall = sum(accept * gold)/sum(gold) According to this model, recall can be improved in at least three ways: r Plan A: Increase a (acceptance rate) r Plan B: Increase r (reviews per paper) r Plan C: Increase g (mixture of good to random reviews) We ought to do all of the above, as much as possible.
as for Plan C, of course, life would be good if reviewers were better than they are.