Word occurrence frequencies have a typical Zipfian nature (Manning et al., 2008), with many rare and few highly frequent terms.
representing the long tail of rare terms with short embeddings should greatly reduce memory requirements.