1 Due to their ability to capture syntactic and semantic information of words from large scale unlabeled text, pretrained word vectors (Turian et al., 2010;Mikolov et al., 2013;Pennington et al., 2014) are a standard component of most state-ofthe-art NLP architectures, including for question answering , textual entailment (Chen et al., 2017) and semantic role labeling .
these approaches for learning word vectors only allow a single contextindependent representation for each word.