Kumar et al. (2018) propose a differentiable surrogate for the expected calibration error, called maximum mean calibration error (MMCE), using kernel embedding.
such a kernel embedding method is computationally expensive and not scalable to the large pre-trained language models.