Minibatch stochastic gradient descent is applied to look for a set of parameters θ * that maximizes the log likelihood of producing the correct target sentence.
given a batch of training pairs {(X i , Y i )}, we aim to find θ * which satisfies: Whereas during inference the model can generate the target sentence one token at a time, advancing time by one step.