However, these multilabel criteria cannot meet our requirement exactly as none of them considers the ranking among emotions which are considered relevant.
by incorporating PRO loss (Xu et al., 2013), the loss function for the instance x i is defined as follows: (1) where e t refers to the emotion belonging to relevant emotion set R i or the threshold Θ of instance x i while e s refers to the emotion which is less relevant than e t denoted as ≺.