However, if exclusion was exploited, both encoder and decoder will recognize that escape from order 1 to order 0 is inevitable because the order 1 model adds no characters that were not already predicted by the order 2 model.
first, some errors are obviously caused by deficiencies in the training data, such as improperly segmented common words and names.