weighted sum of the task-specific loss L task and the auxiliary LM loss L LM , where γ is a weighting parameter to enable adaptation to the target task but at the same time keep the useful knowledge from the source task.
: Exponential decay of γ.