An exponential decay in the number of words that are assigned longer representations is one possible way to implement this.
we propose to have the number of words that receive a trainable parameter at dimension j decrease with a factor α j (α ∈ ]0, 1]).