Mann and McCallum (2007) and following work take the base classifier p θ (y|x) to be a logistic regression classifier, for which they manually derive gradients for the XR loss and train with LBFGs (Byrd et al., 1995).
nothing precludes us from using an arbitrary neural network instead, as long as it culminates in a softmax layer.