We observe notable differences in approaches depending on the nature of the evaluation metric.
deeper or more complex models (which require greater time and resources to train) generally perform best in the supervised setting, whereas shallow log-bilinear models work best on unsupervised benchmarks.