When computing derivatives with backpropagation, these shared dependencies must be taken into account.
the outputs y i of the antecedent resolution layer are the result of a softmax applied to functions of some shared variables q: The derivatives of any y i with respect to q, which can be any of the weights in the anaphora resolution subnetwork, have dependencies on the derivatives of the other softmax inputs with respect to q: This makes the implementation of backpropagation for this part of the network somewhat more complicated, but in the case of our networks, it has no major impact on training time.