Each high dimensional feature in F is induced for making classification between two labels, y andŷ, but it may or may not be helpful for distinguishing labels other than those two.
our algorithm can be modified to learn the weights of the induced features only for their relevant labels by adding the label information to F, which would change the line 13 in Algorithm 1 as follows: introducing features targeting specific label pairs potentially confuses the classifier, especially when they are trained with the low dimensional features targeting all labels.