The most visible effect of the reranker is that the recall is greatly improved.
this does not seem to have an adverse effect on the precision until the candidate set size goes above 16 -in fact, the precision actually improves over the baseline for small candidate set sizes.