The resulting mixture model is deficient (non-normalized), but easy to fix by backing off to a global distribution such as p(o) in equation 1.
we found that this "fix" caused large drops in performance, for instance from the Arabic BLEU score of 48.3 reported in table 3 to 46.0.