Word embeddings have been an essential part of neural-network based approaches for natural language processing tasks (Goldberg, 2016).
many popular word embeddings techniques have a fixed vocabulary (Mikolov et al., 2013;Pennington et al., 2014), i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus.