The RGM must estimate P(w j+1 |w 1: j , S 1:i ), however, since the first i lines have been encoded in the context vector u j i , we compute P(w j+1 |w 1: j , u j i ) instead.
the probability P(S i+1 |S 1:i ) becomes: Let |V | denote the size of the character vocabulary.