Given these definitions, we see that the (M T M ) n term of smaller n corresponds to lower-order cooccurrence, which is accurate but sparse, and the (M T M ) n term of larger n corresponds to higherorder co-occurrence, which is dense but possibly giving too much weight on unrelated instances extracted by generic patterns.
it is expected that setting diffusion factor β to a small value prevents semantic drift and also takes higher order pattern vectors into account.