By formulating such ideas as search or MDL problems of given coding length 1 , word boundaries are found in an algorithmic fashion (Zhikov et al., 2010;Magistry and Sagot, 2013).
such methods have difficulty incorporating higher-order statistics beyond simple heuristics, such as word transitions, word spelling formation, or word length distribution.