Congratulations (bow-theknee emoticon)" Comparing Default with LogReg and LinSVM, we can see that the linear models performed better than the default RNN model without pretraining, when the labeled data size is less than or equal to 20K.
looking at the results of Dial, our method improved Default even for these cases (5K to 20K), and Dial clearly outperformed the linear models.