3 The log probability of tuning data w is which expands according to the definition of p LL n i The gradient with respect to λ i has a compact form where CH is cross entropy.
computing the cross entropy directly would entail a sum over the vocabulary for every word in the tuning data.