Deflated Sharpe ratio and PBO: pricing in your own luck

Why raw Sharpe lies when you try many strategies

Suppose a coin-flip strategy has a true Sharpe of zero. Run one backtest and by chance you might observe 0.5 or 1.0. Run 500 variants and the best one will look genuinely impressive — a 1.5 or 2.0 Sharpe — even though none of them has any real edge. You selected the maximum of a noisy distribution, and the maximum is always flattering.

This is the multiple testing problem, and it is invisible in a single equity curve. Any platform that lets you optimize hundreds of parameter sets and then shows you only the best run is quietly selling you selection bias.

What the deflated Sharpe ratio (DSR) actually does

The deflated Sharpe ratio, introduced by Bailey and López de Prado, takes your observed Sharpe and deflates it by three things: the length of the track record (fewer observations inflate luck), the non-normality of returns (skew and fat tails), and — most importantly — the number of trials you ran to select the strategy.

The output is the probability that the strategy's true Sharpe is positive after correcting for selection bias. A raw Sharpe of 1.5 with 10 trials might deflate to a DSR of 0.7; the same 1.5 Sharpe after 5,000 trials can deflate below 0.1. The trial count is not a footnote — it often decides the verdict.

PBO: the probability of backtest overfitting

PBO asks a slightly different question: if I split my trials into in-sample and out-of-sample subsets and re-select the best strategy on the in-sample subset, how often does that same strategy underperform out-of-sample? The standard implementation is CSCV (combinatorially symmetric cross-validation): strategies are partitioned into blocks, the selection is repeated many times, and the fraction of times the in-sample winner becomes an out-of-sample loser is the PBO.

A PBO above 0.5 means your best result is more likely overfit than real. A PBO below 0.1 is the kind of number an institutional desk would actually look at twice.

A worked example: 500 trials on XAU/USD

Imagine you ran a genetic search that evaluated 500 parameter sets on XAU/USD H1. The best variant reports a Sharpe of 1.8 and a profit factor of 2.1. Before believing it:

• Count the trials honestly: 500 evaluations, plus any parameter tweaks and indicator variants you tried along the way.
• Compute PBO with CSCV across the strategy pool. If the in-sample winner loses out-of-sample in 6 of 10 folds, PBO is 0.6 — not deployable.
• Check the deflated Sharpe with N = 500. A 1.8 raw Sharpe with a 2-year record and 500 trials typically deflates to a DSR well below the 0.95 confidence threshold.
• If DSR and PBO both fail, the honest verdict is: the backtest found a lucky variant, not an edge. Keep searching — with more conservative gates.

How EasyQuant productizes DSR and PBO

In EasyQuant, DSR appears in the defense and significance stages of the robustness pipeline, and CSCV-style PBO summaries are available on library strategy batches. Because the Forge counts every trial, the statistics have real input: the same engine that generated your strategies also reports how many were tried.

Strategy cards carry honesty badges that surface these results instead of hiding them. A strategy can fail the DSR/PBO gate and still be visible — flagged as research-only, with the evidence shown. That is the glass-box principle applied to statistics.