While MIRA (Chiang et al., 2008), Rampion (Gimpel and Smith, 2012), and HOLS (Flanigan et al., 2013) have been shown to be effective over larger feature sets, they are difficult to explicitly regularize -this will become important in Section 4.2.
we use the PRO optimizer (Hopkins and May, 2011) as our baseline learner since it has been shown to perform comparably to MERT for a small number of features, and to significantly outperform MERT for a large number of features (Hopkins and May, 2011;Ganitkevitch et al., 2012).