We use the pairwise ranking optimization (PRO) algorithm (Hopkins and May, 2011) implemented in the open-source Joshua toolkit (Ganitkevitch et al., 2012a;Post et al., 2013) for tuning.
specifically, we train the system to distinguish a good candidate output j from a bad candidate j , measured by an objective function o (section 3.1), for an input sentence i: the optimization reduces to a binary classification problem.