KL Divergence
Also: Kullback-Leibler divergence · relative entropy · KL
Kullback-Leibler divergence measures how far one probability distribution sits from another. It quantifies information lost when you approximate reality with a model — or how far two markets have drifted apart.
KL divergence answers “how different are these two distributions?” — but asymmetrically. It measures the extra information (in bits or nats) you’d need to describe data drawn from distribution P if you encoded it using distribution Q. Zero means identical; larger means further apart. Because KL(P‖Q) ≠ KL(Q‖P), direction matters.
It shows up everywhere in trading ML: as the loss that trains classifiers to match true label distributions, as a regularizer that keeps a policy from straying too far from a prior, and as a drift detector when today’s feature distribution diverges from what a model was trained on.
A concrete use for an agent: applied to related markets, KL surfaces pairs that have drifted apart — candidate mean-reversion or relative-value trades. More defensively, an agent can monitor the KL divergence between live data and its training distribution; a spike is an early warning that the model is now operating out-of-sample and its edge may have evaporated.
- probability
- information theory
- pairs
Research source: rSwarm research library →