If δ < d, we can first project word embeddings to a lower δ-dimensional space using some dimensionality reduction method such as SVD, whereas if δ >d we can learn higher δ-dimensional overcomplete word representations  from the original d-dimensional word embeddings.
we will limit our theoretical analysis to the δ = d case for ease of description.