The idea of using the regularizer of a linear model to encourage the target parameters to be close to the source parameters has been used previously in domain adaptation.
chelba and Acero (2004) showed how this technique can be effective for capitalization adaptation.