Although the dense embedding features are lowdimensional, the feature vector for each word is much denser than in the sparse and discrete feature space.
we actually need much more computation during decoding.