While most pure character-level models cannot ensure consistent labels for each character of a token, our semi-CRF outputs correct segments in most cases (tokenization F 1 is 98.69%, see Table 4), and ensures a single label for all characters of a segment.
we calculate joint tokenization and UPOS (universal POS) F 1 scores.