Post by Bitcoin Dictionary

But calibration on one dataset does not necessarily transfer to new domains, adversarial prompts or long reasoning chains. Token probability is not the same as claim correctness. Verbal confidence can be shaped by style training. Retrieval introduces another failure layer: a model can retrieve an authoritative source and still misinterpret it.

A complete system needs:

Claim-level rather than token-level uncertainty.
Separation of epistemic and aleatoric uncertainty.
Source provenance.
Contradiction detection.
Reliable abstention.
Active information seeking.
Explicit tracking of assumptions.
Verification proportional to consequence.
Confidence that remains calibrated after tool use and multi-step reasoning.

Recent work demonstrates that calibration can be trained separately from raw factual accuracy, which is promising but also illustrates that capability and epistemic honesty are different variables. Behaviourally Calibrated Reinforcement Learning

A solution would be an AI whose reported uncertainty remains predictively meaningful across unfamiliar domains, which knows when further evidence is required and which does not convert uncertainty into fluent invention.