They are obtained by using the bidirectional approach with masked language model (Taylor, 1953) in training.
in spite of these improvements, the proposed Transformer Encoder model achieves the highest scores in all metrics for tweet classification even though it does not use the morphological analysis.