The first one, referred to as relaxed evaluation, is commonly used in entity classification tasks (Adel and Schütze, 2017;Bekoulis et al., 2018a,c) and similar to the binary classification baseline system evaluation: score a multibin sub-event as correct if at least one of its comprising bin types (e.g., goal) is correct, assuming that the boundaries are given.
in this work, we propose to use the bin-level evaluation, since it is a more natural way to measure the duration of a sub-event in a supervised sequence labeling setting.