First, unregularized MERT can achieve competitive results with a small set of highly engineered features, but adding a large set of more than 200 features causes MERT to perform poorly, particularly on the test set.
unregularized MERT can recover much of this drop of performance if it is given a good sparse initializer w 0 .