This experiment shows that token boundaries do not only depend on the language, and therefore that applying a certain tokenizer for this sole reason is not always optimal.
the fact that traditional tokenizers are rule-based does not imply that there is a unique way to tokenize a lan-  Table 2: Token-based recall (percentages) when applying a tokenizer trained on a given dataset (rows) to another dataset (columns) in the same language.