Pre-trained word embeddings learned from unlabeled text have become a standard component of neural network architectures for NLP tasks.
in most cases, the recurrent network that operates on word-level representations to produce context sensitive representations is trained on relatively little labeled data.