It is an interesting question whether our techniques for hard clustering can be extended to soft clustering.
soft clustering models tend to be larger than hard clustering models because a given word can belong to multiple clusters, and thus a training instance P(w i |w i-2 w i-1 ) can lead to multiple counts instead of just 1.