weights obtained at each iteration are likely to be best for the metric that was optimized in that iteration.
one could use weights from the last k iterations (for lateen-tuning with as many metrics) and then decode the test set with an ensemble of these weights as in PMO ensemble.