In Sequential Viterbi Models, such as HMMs, MEMMs, and Linear Chain CRFs, the type of patterns over output sequences that can be learned by the model depend directly on the model's structure: any pattern that spans more output tags than are covered by the models' order will be very difficult to learn.
increasing a model's order can lead to an increase in the number of model parameters, making the model more susceptible to sparse data problems.