Since it is impossible to define the complete vocabulary, we prepare a special token ÍÏ for unknown words and an unknown word spelling Ü ¼ ½ is predicted by the following character-based Ò-gram model after ÍÏ is predicted by Å Û Ò : where Ü ´ ¼µ and Ü ¼ •½ is a special symbol Ì.
when Û is outside of the vocabulary Ï, Nagata (1994) proposed a stochastic word segmenter based on a word Ò-gram model to solve the word segmentation problem.