The gradient, as in all energy-based models, takes the form of the difference between two expectations (LeCun et al., 2006).
we propose a multi-task objective that jointly predicts the next word w and its morphological tag t given a history h. we are interested in a joint probability distribution defined as where f t is a hand-crafted feature vector for a morphological tag t and S is an additional weight matrix.