The former typically leads to a solution that is sparse and easily interpretable, while the latter leads to a non-sparse, generally more stable one.
the squared 2 and 1 penalties were used in the context of log-linear language models (Chen and Rosenfeld, 2000;Goodman, 2004), reporting performances competitive with bi-gram and tri-gram interpolated Kneser-Ney smoothing.