More precisely, define the set Z D to contain all the unique label type sequences appearing as a subsequence of some sequence y (m) ∈ D. Then, if we're willing to tolerate some errors in reconstructing y, we can use a dynamic program to minimize the number of mislabelings in our now "prediction"ŷ, plus the number of distinct segments used in formingŷ multiplied by a constant c, as follows: where J(0) = 0 is the base case and |z| is the length of sequence z.
in Figure 3 we plot both the F 1 score and the average number of distinct segments used in predicting eachŷ against the c parameter from the dynamic program above, for the CoNLL 2003 NER development data in both the standard and zeroshot settings.