Pre-trained Transformers are now ubiquitous in natural language processing, but despite their high end-task performance, little is known empirically about whether they are calibrated.
do these models' posterior probabilities provide an accurate empirical measure of how likely the model is to be correct on a given example?