Empirically, online methods are of-ten faster by an order of magnitude (Collins et al., 2008), and it has been argued on theoretical grounds that the fast, approximate nature of online methods is a good fit given that we are interested in test performance, not the training objective (Bottou and Bousquet, 2008;Shalev-Shwartz and Srebro, 2008).
in the unsupervised NLP literature, online methods are rarely seen, 5 and when they are, incremental EM is the dominant variant (Gildea and Hofmann, 1999;Kuo et al., 2008).