In the following, we consider as a "bigram" a language model with a temporal history that includes information from no longer than one previous time-step into the past.
as can be seen, the FLM alone might increase perplexity, but the GPB-FLM decreases it.