It is important to have a method at hand that gives us assurances that the observed increase in the test score on a test set reflects true improvement in system quality.
we need to be able to gauge, if the increase in score is statistically significant.