Therefore, the embeddings of a sentence are represented as l×(dp+dw) .
we adopt one-hot encoding to embed positions of tokens, rather than sinusoidal position encoding recommended in BERT model (Devlin et al., 2018).