Thus, the transferred information is diluted while we train the target model with more data.
our transfer method explicitly saves the transferred information in the base part of our target model, and we use separate learning rates to help the target model to preserve the transferred knowledge.