The pre-training data encoding such strong correlation will makes the models ignore the context utterance and convergence to the local optimum only related to R 0 .
the balanced pre-training dataset is effective in initializing the networks.