While all of the above-mentioned research has presented intrinsic evaluations of their disengagement modeling efforts that indicate a reasonable degree of accuracy as compared to a gold standard (e.g., manual coding), only a few have yet demonstrated that the model's detected values are useful in practice and/or are a reasonable substitute for the gold standard with respect to some practical objective (e.g., a relationship to performance).
two studies (Bohus and Horvitz, 2009;Schuller et al., 2009a) have gone directly from intrinsic evaluation of (dis)engagement models to performing user studies with the implemented model, thereby bypassing other less expensive and less labor-intensive means of extrinsic evaluation to quantify their model's usefulness-and potentially indicate its need to be further improved-before deployment with real users.