Markov bi-gram model looks the category of the target word and categories before/after the target words.
given a word sequence w1 ,...wn , it calculates the probability of each candidate category ck for a target word wk by P(ck I ck_1 ) P(e lc+, I C k) P(wk I ck ).