The final output is obtained by running a softmax function after a similar linear transformation, W 2 , to the hidden vector as follows: where W 2 ∈ R |T |×H 2 , b 2 ∈ R |T | , and y t ∈ R |T | .
b 2 denotes a bias vector, and y t denotes the distribution vector for each possible label.