Coupled with embedding approaches (Mikolov et al., 2013;Le and Mikolov, 2014;Pennington et al., 2014), neural networks can find the optimal feature combinations using techniques such as random weight initialization and back-propagation, and have established the new state-of-the-art for several tasks (Socher et al., 2013;Devlin et al., 2014;Yu et al., 2014).
neural networks are not as good at optimizing combinations between sparse features, which are still the most dominating factors in natural language processing.