This removes unconditionally probable (frequent) words from the top LM predictions.
each representative is a set of size 2 , containing samples from the forward distribution and samples from the backward distribution.