On the other hand, comparing each two documents is a costly and time-consuming task, especially as the number of documents grows.
it is important to find a balance between the scale of redundancy detection comparisons and the degree of redundancy that is acceptable in the resulting corpus.