Our work is motivated by the goal of assisting human moderators of online communities by preemptively signaling at-risk conversations that might deserve their attention.
we caution that any automated systems might encode or even amplify the biases existing in the training data (Park et al., 2018;Sap et al., 2019;Wiegand et al., 2019), so a public-facing implementation would need to be exhaustively scrutinized for such biases (Feldman et al., 2015).