It is well established that textual information can be predictive of age, race, gender, and many other social factors of the author (Koppel et al., 2002;Burger et al., 2011;Nguyen et al., 2013;Weren et al., 2014;Verhoeven and Daelemans, 2014;Rangel et al., 2016;Blodgett et al., 2016), or even the audience of the text (Voigt et al., 2018).
any system that incorporates raw text into its decision process is at risk of indirectly conditioning on such signals.