A context n that is too small will not capture sufficient information to accurately model character dependencies.
a context n that is too large will create sparse data problems in training Both extremes will result in a larger perplexity than an optimal length.