Learned or memorised? How to tell whether your strategy has a real edge

Fingerprint one: the result depends on an exact value

Take any parameter in your strategy — an RSI period, an ATR multiplier, a stop distance — and move it slightly. A learned edge usually degrades gradually, because the underlying behaviour is smooth. A memorised edge often collapses the moment you step off the exact value, because the number was tuned to the accidents of your particular sample.

The test costs five minutes: run the strategy at the chosen value, then at ±20%, and plot the three results. If the middle one is a spike rather than the top of a hill, you have your answer.

Fingerprint two: the result depends on a few trades

Remove the single best trade and re-run. Then remove the best two. A strategy with hundreds of trades and a real edge will barely notice. A strategy whose entire profit comes from a handful of outlier wins is not necessarily broken — trend following genuinely works that way — but you should know which one you are holding, because the confidence you can place in it is different.

A related check: look at the profit contributed by the top 5% of trades. If it is more than half of the total, the strategy's result is a statement about a small number of events.

Fingerprint three: the result depends on the market you chose

Run the same rules, unchanged, on a different but related instrument. A real edge usually leaves a faint trace elsewhere: weaker, sometimes unprofitable after costs, but recognisably present. A memorised pattern usually vanishes entirely, because the accidents it captured were specific to that one price series.

This test is demanding, and failure is not proof of overfitting — some genuine edges are market-specific. But success is meaningful evidence, and it costs you an afternoon.

Fingerprint four: you cannot explain why it works

This is the softest sign, and the one most often dismissed. If you cannot state in one sentence what behaviour the strategy is exploiting and who is on the other side of the trade, that is information. It does not prove the strategy is fitted, but it means you have no way to know when the reason stops applying.

A concrete version: can you say whether the edge comes from risk transfer, from a liquidity premium, from a behavioural bias, or from a structural constraint? If the answer is 'it just backtests well', you are holding a curve, not a thesis.

Fingerprint five: you tried a lot of things

This one is about you, not the strategy. If you tested two hundred variations and kept the best, the winner's Sharpe ratio is inflated by the search itself. Statistical corrections such as the Deflated Sharpe Ratio exist precisely to adjust for this, and the adjustment can be severe when the number of trials runs into the thousands.

A practical discipline: write down how many configurations you tried, before you look at the results. That number is part of the result, and any honest report of it should include it.

The uncomfortable conclusion

None of these tests can prove that a strategy will work. They can only make it harder to fool yourself, which is a smaller and more achievable goal. The traders who last are not the ones with the best detection method; they are the ones who assume they are overfitting until several independent checks say otherwise, and who size positions so that being wrong stays survivable.