Prior studies show that the encoders trained in this way can capture generic contextual information of the input text and improve a variety of downstream tasks significantly.
it is rather a waste to allocate extensive computational resources to the softmax layer.