One might be surprised the improved results by KVQ-FH since the model sizes of KVQ-FH were relatively very small comparing with the original embeddings.
this may be a reasonable observation since our method additionally offered the embeddings of OOV words that cannot be handled by the original embeddings.