Efficient model-memory joint training is beyond the scope of this paper and can be investigated in future work.
the parameter β could be optimized to balance the contribution from the model part and the memory part, but constrains have to be carefully settled to avoid overfitting to the training data.