Independently of its explicitness, some of this private information correlates with the output labels, and therefore will be learned by the network.
improving the privacy of the neural representations comes at a cost in accuracy.