Despite intensive research efforts to remove noise automatically (Han and Sun, 2016;Hoffmann et al., 2011;Jiang et al., 2016;Surdeanu et al., 2012;Zhou et al., 2016), this problem has yet to be solved.
some groups (Angeli et al., 2014;Liu et al., 2016;Pershina et al., 2014;Zhang et al., 2012) have attempted to boost the performance of relation classifiers through crowdsourcingbased training data collection for relation extraction and using them along with training data for DS learning.