Applied to our prob-lem, the objective becomes: (2) In order to use the previously defined objectives, we need to compute the model probabilities p θ (y|x), which are usually obtained using a softmax function: applied on scores (or logits) s θ (x, y ) output by the model for every possible target token y ∈ Y.
as explained earlier, Y can be very large, hence computing all the scores and summing them is extremely slow.