When we only pretrain the compressing model, the large portion of words, around 80%, is assigned to the largest code-book (i.e., 128).
when we fine-tune the pre-trained models to the task, the ratio of the large one is significantly decreased.