Python backtesting libraries: the trade-off that matters

The one real fork: vectorised versus event-driven

Almost every Python backtesting library sits on one side of a single design choice.

**Vectorised** frameworks compute signals across an entire array at once and derive the return series arithmetically. They are fast — a decade of minute bars in seconds — and concise, and excellent for screening many idea variants. Their structural weakness is that they compute positions rather than orders. There is no order lifecycle, so there is no natural place for an execution delay, a partial fill, or a stop and target that both trigger inside the same bar. Those get approximated or omitted, and the omission is optimistic.

**Event-driven** frameworks process a stream of bars or ticks and dispatch orders, which the framework fills according to its model. They are slower and more verbose, and they have somewhere to put the realism: you can delay a fill by one bar, fill a stop at the stop price plus slippage, and handle a rejected order. Their weakness is that the realism still depends on how you configure it — an event-driven framework with zero slippage and fills at the signal bar's close is just as optimistic as a vectorised one.

The practical consequence: if you are screening hundreds of variations, vectorised is the right tool and you should treat its output as a ranking rather than a result. If you are deciding whether to risk money, you need an execution model, and that means event-driven or a hand-written loop.

What the mainstream options are for

Rather than a ranking, which would depend entirely on your constraints, here is what each family is designed to do.

**Backtrader** is the long-standing general-purpose event-driven framework: broker simulation, order types, multiple data feeds and indicators. It is expressive and well-known, and its development is comparatively quiet these days, so check that it suits your Python version before committing.

**Zipline** and its community fork originated in an institutional equity context, with a pipeline model for cross-sectional factors. Strong for equity universes, heavier to set up for a single instrument on a broker feed.

**vectorbt** is the vectorised extreme: very fast parameter sweeps over large arrays, designed to explore thousands of combinations quickly. Excellent for that purpose, and it leaves execution realism to you.

**Backtesting.py** is a compact, readable single-asset framework with a low learning curve. Good for learning and for smaller studies; less suited to multi-asset portfolios.

**bt** focuses on portfolio-level backtesting of weightings across assets, which is a different problem from trading one instrument with stops.

**pandas and numpy by hand** is a legitimate option and often the right one. A loop over bars with explicit costs is perhaps a hundred lines, has no dependency you have to keep alive, and makes every assumption visible because you wrote it. The failure mode here is the opposite of a framework's: nothing is hidden, but nothing is provided either, so it is easy to omit the execution delay simply because you did not think of it.

The questions to ask before adopting any library

**Can I delay a fill by one bar?** If the framework fills at the close of the bar that generated the signal, and you cannot change that, you have a systematic optimistic bias you cannot remove. This is the single most important question.

**How does it resolve a bar where my stop and target are both touched?** Look for a documented answer. If there is none, the behaviour is whatever the implementation happens to do, which is not a specification.

**Where do costs enter?** Per fill, with a direction, or as a single deduction at the end? Per fill is right. A deduction at the end gives the same total but hides which trades the costs killed.

**Does position sizing see the equity curve?** Fixed-fractional sizing requires the position size to depend on current equity, which requires the sizing logic to run inside the simulation rather than being applied afterwards.

**Can I export the trade list?** You will want to compare trades rather than summary statistics, because two runs can produce the same final return from different trades. A framework that will not give you the trade list is not usable for serious work.

**What happens when data has gaps?** Silent gap-filling manufactures price action that never happened. You want the framework to tell you, not to interpolate quietly.

The mistake that matters more than the choice

The largest source of error in a Python backtest is not the library. It is running the framework at its defaults and reading the output as a measurement.

Defaults exist to make the software easy to start with, and easy to start with means assuming no costs and ideal fills. A framework that shipped with a pessimistic default would produce discouraging first results for everyone, and nobody would use it. So the defaults are optimistic by design, and the responsibility for changing them is yours.

The discipline that catches this does not depend on the library at all. Run the backtest, then double every cost assumption, keep the signals identical, and run it again. If the strategy survives with a smaller but still positive result, the edge exceeds the cost uncertainty. If it collapses, you found out for free.

Then compare trade lists rather than final returns. The return can coincide by accident between two runs; the list of trades cannot. This is the check that tells you what actually changed when you moved a setting.

What no library will do for you

No library can tell you whether your idea has an edge. It computes what you specify.

No library can fix a sample that is too small. A beautiful framework running a strategy with forty trades still has forty trades of evidence.

No library removes the gap between simulation and live execution. Broker rejections, disconnects, partial fills and re-quotes are all absent from a bar series, and all of them happen.

And no library protects you from testing so many variations that the best one is the luckiest one. That is a property of your process, and switching frameworks changes nothing about it.