For example, we do not need to split a number like 1, 000.54 into more units whereas we need to split a comma-separated list of words.
tokenization is important as it reduces the size of the vocabulary and improves the accuracy of the taggers by producing similar vocabulary to the one used for training.