LEARN · EN · easyquanttrading.com
We generated 73,115 strategy candidates. Here is what that does to the ones that looked good.
The most dangerous number in quantitative research is not a bad backtest. It is a good one, produced by an idea that was tested after many other ideas failed. We can show you our counts because we keep them.
By the EasyQuant Research Team·Published 2026-09-26·We publish the tests our own strategies fail. Nothing here is a return promise.
- Testing many ideas manufactures false positives at a predictable rate
- The correction has to be applied before you look, not after
- Candidate count and surviving count are different numbers and should be reported separately
- A shelf that never shrinks is a shelf that is not being checked
Our numbers, stated plainly
Across 11,568 discovery runs, our engines produced 73,115 raw candidates. After deduplication, 4,008 distinct strategies were stored for evaluation.
Today, 457 are on the shelf. 1,389 have been retired. 203 were removed specifically for going negative out-of-sample. 241 were removed purely as duplicates of something already held.
We report the generated number and the surviving number side by side, because quoting only one of them is how a research process flatters itself.
Why the raw count is not a scoreboard
Every additional idea you test raises the best-looking result you will observe, even if none of the ideas has any merit. This is not a quirk of markets; it is what taking a maximum over more samples does.
So a discovery engine that reports only its best result is reporting a maximum, and a maximum without the number of attempts is not interpretable.
This is why we keep the counts at all. Without them, our own best result would be unfalsifiable to us as well.
What we do about it
Before a strategy is judged, it is measured out-of-sample: fit on one part of the history, judge on another. A strategy that only works on the part it was fitted to is discarded, however good that part looked.
We also report validation checks separately — signal health, out-of-sample behaviour, walk-forward, Monte Carlo, deflated Sharpe, probability of backtest overfitting — and failures stay visible rather than being averaged away.
None of this makes a strategy correct. It makes it checkable, which is the most that can honestly be offered.
The uncomfortable implication
If your process does not track how many ideas it tested, you cannot know how much of your best result is selection.
The fix is not a better backtester. It is keeping the count.
Current platform facts
Read live from the strategy library when this page was generated. These are the same counts published on our transparency page, and they change as strategies are added and rejected.
| Strategies in the audited library | 3672 |
|---|---|
| Flagged by the audit | 2011 |
| Flag rate | 54.8% |
| Checks still pending | 1651 |
| Passed the DSR overfitting check | 1 |
| Passed the significance check | 504 |
| DSR threshold used | 0.90 |
FAQ
- Does this mean good strategies do not exist?
- No. It means a good-looking result is not evidence on its own. The evidence is the result surviving tests that were specified before the result was seen.
- Why publish numbers that make your own shelf look emptier?
- Because the alternative is a shelf we cannot defend. A number we can defend and a larger number we cannot is not a close call.
- Is 203 out-of-sample failures a lot?
- It is the expected consequence of automated search over a large parameter space. The number that would worry us is zero, which would suggest the test is not being applied strictly.
More guides
- How EasyQuant validates strategies — evidence you can filter
- Honest backtesting, not pretty curves
- Gold strategy research that stays honest
- Overfitting detection: catch it before you deploy
- System Forge: design, then prove
- Walk-forward analysis: the only backtest that fights overfitting
- MT5 export without custody
- Glass box, not black box AI signals
Not investment advice. Historical results do not guarantee future performance. EasyQuant is a research factory — you execute on accounts you control.