All models except the 4th are learned via L2 penalized ("ridge") regression (Goeman et al., 2016 Residualized Control Approach In order to effectively exploit Twitter language in our model, we suspect that we need to treat the language features (which are numerous, noisy, more biased, and non-normal) differently than the control variables (which are few, mostly unbiased, and mostly normal).
simply combining the two may lead to losing the importance of the controls amongst the numerous features.