We performed a grid search over (C, classifier) ∈ {0.5, 1, 5, 10, 50, 100} × {one-vs-one, one-vs-rest} of linear SVM using the train set for each vector representation, and 2 We used {ja,zh}wiki-20170220-pages-articles1.xml in https://dumps.wikimedia.org 3 In this experiment, we defined the frequent n-grams as the union of the top-kn frequent ngrams, where n and kn are the pre-specified numbers.
word segmentation, that often requires manually annotated resources, is difficult and expensive, and unavoidable errors in word segmentation affect downstream tasks.