Do large language
models know when
they are wrong?

Calibration is the match between how confident you sound and how often you are right. A perfectly calibrated forecaster who says "80% confident" is correct on about 80% of those answers. Not 95%. Not 60%. Eighty.

That is a separate skill from accuracy, and it is the one that decides how much your confidence is worth to anybody else. A model that is right 92% of the time but claims 99% is not lying about the answer — it is lying about how hard the question was. If you are a nurse or a loan officer reading that number to decide whether to check the work, that second lie is the one that costs you.

The standard measure is Expected Calibration Error (ECE). Group every answer by its stated confidence, ask how often each group was actually right, and average the gaps. Zero means perfect. In the chart below, perfect calibration is the brass diagonal — say 70%, be right 70% of the time. Above the line you are underconfident; below it you are overconfident.

The humility prompt worked. That is what made it worse.

Averaged across both models and all three domains, the humility prompt significantly lowered stated confidence relative to accuracy b = −0.05, p = .004. The instruction did exactly what it said on the tin.

And calibration got worse for both models. Claude's ECE went from .047 to .100; GPT's from .053 to .066. Because "be humble" is not a calibration instruction — it is a direction. It pushes confidence down whether or not down is where you needed to go. GPT started above the line, so down helped its average gap. Claude started below the line, so the same sentence pushed it further from the truth.

Making a model sound less confident is not the same as making it more honest.

And the expert persona did nothing at all.

We predicted that telling a model to answer "as a leading expert" would inflate its confidence without improving its accuracy. It didn't. The effect was b = 0.002, p = .925 — not a noisy result we failed to resolve, but an estimate sitting almost exactly on zero. That is an informative null, and it runs against Xu et al. (2025), who found expert framing did inflate confidence. Our instruction may simply have been too mild; Z. Zhou et al. (2025) found that only strongly worded steering reliably moves confidence.

None of it depended on the subject.

We expected prompts to bite hardest in medicine and finance, where the stakes are highest. They didn't. The prompt-by-domain interaction was χ²(4) = 0.24, p = .993 — about as flat as a null can look. Whatever the prompts did, they did identically to psychology, health, and finance questions.

Calibration asks whether confidence sits at the right level. A separate question — resolution — asks whether a model gives higher confidence to the answers it actually gets right, whatever the level. We measured it with a Goodman–Kruskal gamma correlation between confidence and correctness.

Resolution was clearly above zero in every condition, and every confidence interval excluded zero. Both models could tell their right answers from their wrong ones. GPT was the better discriminator in all three conditions, though the intervals overlap enough that we call that suggestive rather than established.

So the signal is real. The scale it is printed on is off. That is a much more workable problem than noise — and a much more dangerous one, because a number that is almost right is the kind people stop checking.

Every claim on this page about overconfidence rests on the questions the models got wrong. You cannot be overconfident about an answer you got right. So how many wrong answers are we actually working with?

Seven. Everything this study says about GPT's overconfidence rests on seven questions.

Overall accuracy was 91.8%. That is a ceiling effect, and it is the single biggest limitation here. With six to nine errors per cell, there is very little room for a prompt or a domain effect to show up even if one exists. Our null results are consistent with "prompting doesn't work," and they are equally consistent with "we could not have seen it if it did."

There is a likely reason for that ceiling, too. MMLU-Pro is a public benchmark, so its questions may well have appeared in the models' training data. We may have measured how confident a model is about material it has already seen — which is not the situation anyone actually worries about.

And one more thing worth saying out loud: eleven of the 89 items account for every error in the entire study. If a handful of those items are ambiguous or badly keyed — which is a normal thing for benchmark questions to be — the error set is not just small, it is fragile.

Everything on this page has been aimed at the machine. But calibration is not an AI metric that psychologists borrowed — it is a psychology metric that AI research borrowed, and the human literature is decades older. Overconfidence is one of the most reliably replicated findings in judgment research. People say 90% and are right 70% of the time, and they do it consistently enough that you can plan around it.

Which is why the number you just generated about yourself matters as much as the ones we generated about the models. When a person tells you they are certain, that word is doing the same job the model's "96%" was doing: it is a request to stop checking. And it is worth exactly as much as that person's calibration, which almost nobody has ever measured — including about themselves.

We are getting good at auditing what machines know. We have never really audited what confident people know.

Cash et al. (2025) ran the direct comparison and found LLM calibration to be broadly comparable to human metacognition — varying by domain and by model, still skewing overconfident, but not obviously worse than us. That result gets read as reassuring. It is at least as easy to read the other way: the standard we are holding these systems to is a standard we have not met, and increasingly the two are feeding each other. Steyvers et al. (2025) found that people reading model explanations systematically overestimated how accurate the model was — and that aligning the model's wording with its real confidence shrank that gap. Two miscalibrated parties, each reading the other's certainty as evidence.

That is the practical argument for measuring human confidence with the same instrument we point at models. Not because people are the problem, but because a confidence score only means something once somebody has checked what it is worth — and we have spent far more effort checking the machine's than our own.

Hypotheses, and how they landed

• NOT SUPPORTED H1 — the humility prompt would improve calibration. It significantly lowered the confidence gap (b = −0.05, p = .004) but ECE rose for both models, so confidence and accuracy ended further apart, not closer.
• NOT SUPPORTED H2 — the expert persona would worsen calibration. Effect was essentially exactly zero (b = 0.002, p = .925).
• NOT SUPPORTED H3 — prompt effects would differ by domain. χ²(4) = 0.24, p = .993. Prompt × model was also null, χ²(2) = 2.15, p = .341, as was the three-way interaction, χ²(4) = 0.68, p = .954.

Three predictions, three failures to confirm — one of them because the prompt moved confidence in the predicted direction without improving the thing the prediction was about. We have reported the contrasts as planned rather than reframing around whatever turned out significant.