Furthermore, we have observed that despite the normalisation, the performances of the pre-trained classifiers were still much better than the randomly initialised ones.
to make them more competitive, we propose to start with optimising only the randomly initialised units while freezing the pretrained ones, then, launch the joint training.