A backtest runs a written rule over past market data to see what the rule would have done. That is all it does. It cannot tell you what the rule will do next, and it is easy to build one that looks good only because it was shaped to fit the past.
Our own lab runs hundreds of pre-registered studies and publishes the ones that fail. The habits below come from that work. They are about method, so this page contains no results.
03.1Write the rule before you look
Write the entry, exit, stop, time window and position size in plain words before you run anything. Then write what result would make you reject the idea. A test whose pass line is decided after seeing the numbers is not a test.
03.2Backtesting in NinjaTrader
NinjaTrader 8 runs backtests in the Strategy Analyzer. You choose the strategy, instrument, date range, bar type and trading hours template, then run a single test or an optimization over a range of inputs. Two settings change results quietly: the trading hours template, and the fill resolution. Stops and targets that sit a few ticks apart can be filled differently on bar data than on tick data, so check which one your test used.
03.3Reading a backtest report
Start with the trade count: a handful of trades says almost nothing. Then look at how results are spread across years and market conditions, the size of the worst losing streak, and how much the outcome depends on a few large trades. Check that commissions and slippage were included.
Then ask how many variations were tried before this one. If fifty settings were tested and the best is shown, the report describes the search as much as the rule.
03.4Walk-forward analysis
Walk-forward analysis splits history into a run of windows. Inputs are chosen on an in-sample window and then applied, unchanged, to the next out-of-sample window. The windows roll forward and the process repeats. Only the stitched out-of-sample segments are judged. It does not remove overfitting, but it makes the cost of re-tuning visible.
03.5Free backtesting software
Several platforms offer free or low-cost backtesting: NinjaTrader's Strategy Analyzer for NinjaScript, TradingView's strategy tester for Pine Script, and QuantConnect's open-source LEAN engine for Python and C#. They differ in data, fill modeling and language. Pick the one whose language you can read and whose fill assumptions are documented.
Where it usually goes wrong
- Changing the rule after every run until the curve looks right.
- Testing only one market period and assuming it repeats.
- Leaving out commissions and slippage.
- Trusting tight stops and targets that were filled on bar data.