If the rare words are not initialized properly, they would also bias the whole word representations.
a model without OOV mechanism will fail to precisely represent those inevitable OOV words from test sets.