Because of the differences between language model and NLU task objectives, the problem of lack of concern on some key words exists.
in this paper, we propose a method called reverse mapping bytepair encoding, which maps named-entity information and other word-level linguistic features back to subwords during the encoding procedure of bytepair encoding (BPE).