As this marginalization is intractable, we derive a variational inference procedure that allows for efficient approximate inference.
the second difference is that our model is non-Markovian: we sample the i th morphological tag m i from a distribution that depends on all previous tags, using an LStM language model ( §4.1).