As we increased the percentage of request goals in the warm start buffer, the overall success rate decreases and the learning curve becomes less smooth.
the effect of pre-training with very large datasets observed in the experiments is the most surprising result of the paper (Erhan et al., 2010): the early examples determine the basin of attraction for the remainder of the training and the supervised fine-tuning cannot escape from it.