We thus claim that understanding the properties of the task and mapping them to the desired properties of the evaluation measure is crucial when selecting a measure for an intrinsic evaluation.
we propose in section 4 a new framework on the intrinsic evaluation of STS systems by taking the requirements of the target task into account.