In practice, we would like to select model parameters without observing all the test topics.
we ran 10-fold cross-validation experiments on each collection, selecting parameters that maximize MAP on nine folds and evaluating on the remaining one.