Further, among the crowdsourcing approaches, we find that the crowd-grades (Model RR-3) does equivalently well (and sometimes worse) than the model using features derived from the crowdsourced speech (Model RR-4).
when we combine all the features from crowdsourcing including the crowd grades, we find much better prediction accuracy (r = 0.76).