If we do not discriminate the unknown words and assign different unknown words with the same token unk , it would be impossible for us to know what is the exact word that unk represents for in the real test.
when using our proposed unknown word processing method, if the model predicts a unkX as the answer, we can simply scan through the original document and identify its position according to its unknown word number X and replace the unkX with the real word.