So far, the only way to overcome this BPE imperfection, its deterministic nature, was to create another subword segmentation algorithm (Kudo, 2018).
we show that BPE itself incorporates the ability to produce multiple segmentations of the same word.