Our primary technical contribution is the introduction of a tool for reporting validation results in an easily interpretable way: expected validation performance of the best model under a given computational budget.
3 given a budget sufficient for training and evaluating n models, we calculate the expected performance of the best of these models on validation data.