For the count models, PMI is clearly the better weighting scheme, and SVD outperforms NMF as a dimensionality reduction technique.
does all of this even matter, or are we on the cusp of discovering radically new ways to tackle the same problems that have been approached as we just sketched in traditional distributional semantics?