Learning maximizes the data likelihood objective with respect to model parameters θ = (W, c): The training procedure uses stochastic gradient descent: it iteratively updates parameters using a gradient at each training subsequence w t−n+1:t .
for the word representations c, the updates have the form: where η is the learning rate and L (t) (θ ) = logP θ (w t |w t−n+1:t−1 ) is the contribution of the example to the data likelihood objective.