With fine-tuning, CNN-SC substantially boosts accuracy and generalization We switch gears  (Zhang et al., 2017) 95.4 98.2 72.6 pre-training + fine-tuning w/o external data CNN-R (Zhang et al., 2017) 96.0 98.8 96.6 99.0 74.9 pre-training + fine-tuning w/ external data ULMFiT (Howard and Ruder, 2018)   now to our fine-tuning experiments.
we take the CNN encoder pre-trained using our sentence content objective and then fine-tune it on downstream classification tasks with supervised labels.