Recently, language models (LMs) or language representation models are widely used in natural language understanding (NLU) tasks.
these LMs are usually trained on large unlabeled text corpora, while the finetuning process simply takes words or wordpieces as model input.