The pooled features are then transformed to predictions over the output vocabulary V, by linearly mapping them with a matrix E ∈ R |V|×f L to the vocabulary dimension |V|, and then applying a soft-max.
the probability distribution over V for the i-th output token is obtained as Alternatively, we can use E to project to dimension d t , and then multiply with the target word embedding matrix used to define the input tensor.