The importance of these knowledge has been verified by tasks at different level, for instance, sentence-level sentiment classification (Taboada et al., 2011;Shin et al., 2017;Lei et al., 2018), aspect-level sentiment classification (Vo and Zhang, 2015;Zeng et al., 2019), opinion extraction (Li and Lam, 2017;Gui et al., 2017;Fan et al., 2019) and so on.
we assume that, by integrating these knowledge into the pre-training process, the learned representation would be more sentimentspecific and appropriate for sentiment analysis.