(2010) show that standard machine learning techniques (in their case, a naive Bayes classifier) can outperfom the Turkers on a categorization task (classifying businesses into Automotive, Health, Real Estate, etc.).
in some cases, NLP tools already do better than MTurk.