Calibration
Also: probability calibration · calibration surface
A forecaster is calibrated when things it says are 70% likely happen 70% of the time. Calibration measures whether stated probabilities match reality — the foundation of honest sizing.
A model can be accurate and still lie about its confidence. Calibration asks a narrower, crucial question: across every time the model said “70%,” did the event happen ~70% of the time? If yes, it’s calibrated; if 70%-labeled events resolve 85% of the time, it’s underconfident, and vice versa.
This matters because sizing depends on probabilities, not just direction. Kelly and expected-value math assume your stated odds are real. Feed them overconfident probabilities and you’ll oversize systematically. In prediction markets, the tradeable edge often is a calibration gap — the market prices 63% where fair value is 72%.
For an agent, calibration is a prerequisite for trusting itself. Before a model’s outputs drive real capital, its probability estimates should be checked against realized outcomes and recalibrated if they drift. An agent that knows its own calibration can size honestly; one that doesn’t will confidently overbet its worst forecasts.
- probability
- forecasting
- prediction markets
Research source: rSwarm research library →