Like one finds in a trade-off between precision and recall, the use of a significance test slightly improved the accuracy of the method for those pairs which it had an opinion about, but also increased the number of pairs which had to be randomly assigned an order.
the net impact of using a significance test for the BNC data was a very slight decrease in the overall prediction accuracy.