As mentioned above, the goal of system adaptation is to maximize the system performance while minimizing the amount of feedback information needed to get the improved performance.
since the feedback provided, and in particular the amount of information therein, is under the control of the user, if the task under study does not naturally lend itself to have this amount of information also controlled by the evaluator, no meaningful comparison between systems can be drawn.