To fight against miscalibration, a natural option is to apply a calibration method such as temperature scaling (Guo et al., 2017) in a post-processing step.
temperature scaling only learns a single parameter to rescale all the logits, which is not flexible and insufficient.