But in general replacing the squared 2 norm with an 1 penalty (over all weights) hampers classification performance (indeed, as we later report, this strategy performs very poorly here).
in our scenario we would like to place a sparsifying 1 regularizer over the contextual (interaction) features while still leveraging the squared 2 -norm penalty for the standard bag-of-words (BoW) features.