Thus, we solve this problem by fixing all the embeddings, except for the word types not found in the pre-training corpus.
instead of leaving the unknown words as the zero vector, we use the labeled data to learn a better representation.