For each source, we used 300 of these annotations and used bagging to find the optimal weights.
we drew 200 samples randomly from the given set of 300 samples and did a grid search to find w i 's and δ such that the Q-METRIC computed using Equation 1 had maximum correlation with human scores.