BERT is a pre-trained language representation model with multiple computation layers that has been shown to improve many NLP tasks.
the sentence (x 1 , x 2 , ..., x n ) would be first fed into the pre-trained BERT model from which the contextualized embeddings of the words in the last layer are used for further computation.