Formally, such models define a matrix P ∈ R d×|V | , which contains d parameters for each word in the vocabulary V .
for a given word type w ∈ V , a column is selected by right-multiplying P by a one-hot vector of length |V |, which we write 1 w , that is zero in every dimension except for the element corresponding to w. P is often referred to as word lookup table and we shall denote by e W w ∈ R d the embedding obtained from a word lookup table for w as e W w = P • 1 w .