Attaining a high accuracy using fully stochastic methods is particularly difficult for Japanese due to the prevalence of orthographic variants (a word can be spelled in many different ways by combining different character sets), which exacerbates the sparse data problem.
since the parser currently uses extremely crude heuristics to rank multiple parses, the first analysis was chosen as the best word-breaking analysis in this experiment whenever the parser returned multiple parse analyses.