Redundant annotation is usually nonproblematic because individual crowdsource judgments are inconsequentially cheap in a class-balanced dataset.
this technique renders the corpus useless for experiments including token similarity (or ngram similarity, semantic similarity, stopword distribution similarity, keyword similarity, etc) as a feature; a machine learner would be likely to learn the very same features for classification that were used to identify the rare-class instances during corpus construction.