We also construct an artificial dataset from PAN16 TWIT where the main task labels are preserved but the demographic label is randomly shuffled (PAN16 RAND), allowing us to run experiments with no PREVALENT or SAMPLE-SPECIFIC gender correlations, only ACCIDENTAL.
the results show that humans had difficulties determining the gender of the author.