The weights of the output layer are also compressed in a same way as embeddings.
the number of trainable neural parameters is a constant regardless of the vocabulary size if we ignore the biases of words.