This is the worst possible scenario for the attacker, as it is training on the exact representations the adversary attempted to remove the protected attribute from.
for example, with 5 adversaries, the main-task accuracy remained at random for 5 epochs, and only begun rising at the 6th epoch.