Although this assumption is not always correct in practice, we consider it a reasonable approximation given what we empirically observed in our training data.
to standard CRFs, semi-Markov CRFs directly model the segmentation of an input sequence as well as a classification of the segments (Sarawagi and Cohen, 2004) In this case, the features f (s j−1 , s j , x) are defined on segments instead of on word tokens.