Backtesting term

Walk-forward / out-of-sample

Walk-forward testing tunes a strategy on one slice of history, then forces it to prove itself blind on the next slice it never saw. An edge that only works on the years it was fitted to memorized noise, not the market.

Walk-forward testing tunes a strategy on one slice of history, then tests it on the next slice it has never seen, and rolls that window forward through time. Fit on one stretch of data. Test blind on the stretch immediately after it. Fold that stretch into the training set, then test blind on the next one. Repeat until the window reaches today.

Out-of-sample is the whole point. If an edge is real, it works on data the optimizer couldn't peek at while it was choosing parameters. If it only shines on the years it was fitted to, it memorized noise and called it a rule. This is the check that separates a genuine rule from a well-decorated coincidence, and it's the one most published backtests skip because it makes results look worse.

Without walk-forward, a strategy only has to satisfy one master: the historical curve it was tuned against. Add enough parameters, a stop here, a filter there, a magic lookback length, and almost any equity curve can be made to climb. None of that tuning has to survive a year the optimizer never touched. Walk-forward forces it to.

We test on years and instruments held out of any tuning, running that held-out window against 13 years of CME futures and tick-level FX data with real bid/ask, not the stretch the strategy was calibrated on. A strategy that needs to see the answer before the test isn't forecasting. It's reciting.

A system that aces the in-sample fit but collapses on walk-forward is usually just overfitting in disguise, a fit found by trying enough variations that one was bound to match the noise. That risk compounds with every extra configuration tried, which is exactly the problem multiple testing corrects for. A placebo test attacks a related question from another angle, whether the edge would survive if the entries were shuffled to random. Walk-forward asks whether it survives time instead.

Real edges are boring that way. They keep working on data nobody fitted them to, without a fresh round of tuning to stay alive. A rule that needs constant refitting to keep working was never a rule. It was a curve being chased.

The research behind this

López de Prado (2018). “Advances in Financial Machine Learning.” Wiley. — López de Prado's standard reference on backtest overfitting, cross-validation, and avoiding data leakage in exactly this kind of held-out testing.
Bailey, Borwein, López de Prado & Zhu (2014). “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance.” Notices of the AMS 61(5). — Shows how trying enough parameter configurations manufactures a great-looking backtest even with zero real edge, the exact trap walk-forward guards against.
Harvey, Liu & Zhu (2016). “…and the Cross-Section of Expected Returns.” Review of Financial Studies 29(1). — Shows that after mining enough factors, the bar for calling one real must rise sharply, the multiple-testing logic that motivates testing out of sample.

External research, linked for context and further reading. FoxAlgo is independent and not affiliated with these authors or publishers.

These are the terms behind The No List — the full audit, every strategy and indicator named, with its verdict and the exact reason it lived or died.

Get The No List →