Combinatorial Purged Cross-Validation (CPCV)

Standard k-fold cross-validation assumes samples are independent. Financial samples are not — labels are built from overlapping windows, so a naive split leaks future information into training and reports a backtest far rosier than reality. This is how strategies “work” in research and die in production.

Combinatorial Purged Cross-Validation fixes both problems. Purging removes training samples whose label windows overlap the test set (with an embargo gap after each test block to kill residual leakage). The combinatorial part evaluates many train/test group combinations, producing a distribution of backtest paths instead of one lucky number — so you can see the variance of your edge, not just its best case.

For an agent that rewrites and redeploys its own strategies, this is the guardrail against self-deception. Any candidate strategy an autoresearch loop proposes should clear CPCV before it touches capital, and it should be judged on the distribution of outcomes. An edge that only survives on one favorable path is noise wearing a Sharpe ratio.