This can be viewed as a "soft" version of the maxmargin objective used by  for training without exploration, but retains a locallynormalized model that we can use for samplingbased exploration.
given an arbitrary cost function ∆ comparing structured outputs (e.g.