Each p-value is compared to a threshold which is smaller or equal to α and depends on the number of evaluation datasets N. The dependence of the thresholds on N can be intuitively explained as follows: the probability of making one or more erroneous claims may increase with N, as demonstrated in Section 3.2.
in order to bound this probability by a pre-specified level α, the thresholds for p-values should depend on N. It can be shown that the Holm procedure at level α always rejects thek Bonf erroni hypotheses with the smallest p−values, wherek Bonf erroni is the lower bound for k with a confidence level of 1 − α. ,k Bonf erroni corresponding to a confidence level of 1 − α is always smaller or equal to the number of datasets for which the difference between the compared algorithms is significant at level α.