Walk-forward analysis: in-sample vs out-of-sample, step by step
On this page
Key takeaways
- One round: tune the settings on a training window, then apply them unchanged to the next, unseen test window. Step forward and repeat; the test windows join into one record of results the tuning never saw.
- The gap between training and test results measures how much of the tuned performance was fitted to the training data. Some decay is normal; a large one says the tuning found noise.
- The fair comparison is not the training figure but a simple alternative over the same test months: a fixed setting chosen in advance, or buy-and-hold.
- Rolling training windows forget old data; anchored ones keep everything since the start. Window lengths are themselves choices, and trying many of them brings back the overfitting the method is meant to expose.
- MarketHeist runs two versions: a monthly one in this lab that chooses a trend filter's length for a portfolio, and the backtester's daily one, which differs in where the windows start, how the decay is measured and how each window begins trading.
Definition
Walk-forward analysis tests a strategy's selection rule rather than a single backtest: it chooses the settings on a training window, trades them on the following test window, then moves both windows forward and repeats. Stitched together, the test windows form an out-of-sample record of what choosing settings this way would have produced, decision by decision, with only data available at the time.
Also known as: walk-forward optimization, walk-forward testing, WFO, rolling out-of-sample test, in-sample vs out-of-sample, walk-forward efficiency, anchored walk-forward.
How it works
A backtest with optimized settings answers the question of which settings would have looked strongest, looking back. Walk-forward analysis asks the question an investor faces: if I had chosen settings by this rule at each point in time, using only the data I had, how would the next stretch have gone? Robert Pardo set out the procedure for trading systems in 1992 and expanded it in 2008, and it has become a standard step in validating a systematic strategy.
The mechanics are a loop. Fix a training length and a test length, say ten years and three years. On the first ten years, run every candidate setting and keep the one with the highest in-sample score. Trade that setting, unchanged, over the next three years and record the result. Slide both windows forward by three years and repeat until the data run out. Joined end to end, the test windows form a continuous out-of-sample record. With rolling windows the training period keeps its length and drops old data; with anchored windows it always starts at the beginning and grows.
Two numbers matter. The first is decay: how far the out-of-sample score fell below the in-sample score of the settings chosen. In-sample scores are inflated by selection, because the winner of each training window is partly the luckiest, so some decay is expected. Pardo's walk-forward efficiency, out-of-sample annual return over in-sample annual return, expresses the same idea. The second, and more telling, is the comparison with a rule that needed no optimization, run over the same test months: if a fixed setting chosen in advance did as well, the optimization added nothing.
Walk-forward analysis is like a driving test taken in traffic rather than on the practice course: the course is where the driver was trained, and only the open road shows what was learned.
It has limits of its own. The out-of-sample record is a single path through history, and with long windows it contains only a handful of independent test periods. The training and test lengths, the score and the candidate settings are all choices, and running the analysis many ways and keeping the most flattering version brings the selection problem back. López de Prado (2018) proposed combinatorial cross-validation partly for that reason.
Formula
- k
- fold number, starting at 0, for every fold whose test window ends inside the data
- T_r
- training length in months (120 in the example)
- T_s
- test length in months (36 in the example); folds step forward by this much
- [x, y)
- months from x up to but not including y, counted from the lab window's first month
- N
- the setting being chosen: the trend filter's length in months
- [a, b]
- the candidate lengths (2 to 18 months in the example)
- N*ₖ
- the length with the highest Sharpe ratio on fold k's training window, traded on its test window
- K
- number of folds
- S_IS
- average, over the folds, of the chosen length's Sharpe ratio on its training window
- ⨁ₖ r_testₖ(N*ₖ)
- the monthly returns of each fold's test window under its chosen length, joined in order
- S_OOS
- Sharpe ratio of that joined out-of-sample record
- decay
- the share of the in-sample Sharpe ratio lost out of sample (negative if out-of-sample was higher)
The backtester reports degradation differently: the mean in-sample score minus the mean out-of-sample score over the windows with both, a difference in the chosen metric rather than a share.
- R_OOS, R_IS
- annualized return of the out-of-sample record and of the in-sample optimizations
A return-based version of the same idea. MarketHeist does not report it.
Worked example
The Bogleheads three-fund portfolio (US stocks, international stocks and US bonds in roughly equal parts, rebalanced yearly) with the whole-portfolio trend filter, its length chosen from 2 to 18 months on rolling 120-month training windows and tested on the following 36 months: 6 folds whose test windows run from Jun 2008 to Sep 2026.
In each training window the length with the highest Sharpe ratio was chosen, and the choices moved around from fold to fold: 10, 16, 8, 8, 2, 8 months. On their own training windows those choices averaged a Sharpe ratio of 0.93.
Out of sample, the joined test windows had a Sharpe ratio of 0.54, a decay of 42.5% from the in-sample figure. Keeping the 10-month length throughout, with no optimization, did better over the same test months, a Sharpe ratio of 0.78, and so did simply holding the portfolio, 0.64. Here the search added noise rather than information: the length that had led over one ten-year stretch was not a reliable guide to the next three years.
The filter still did what trend filters do. Over the test months, the walk-forward version's worst decline from a peak was -22.4%, against -30.1% for holding the portfolio, and it paid for that in growth: 3.9% a year against 6.9%.
With only 6 test windows the sample is small, and the folds share most of their training data, so this result is one path through one history. It shows that choosing the length by recent Sharpe ratio was unreliable for this portfolio over these years, not that it never works; switch the lab to anchored windows, other lengths or another portfolio to see how much the answer moves.
Data through Sep 2026. Reproduce id 426debcb3ac87c01 (this guide, its lab setup and the data month).
Interactive lab
Growth of $10,000 over the joined test windows: the walk-forward choice, the fixed 10-month filter and buy-and-hold, with each fold's training and test span and its chosen length. Change the windows, anchor them, or switch portfolios.
| Measure | Walk-forward | Fixed 10 months | Buy and hold |
|---|---|---|---|
| Sharpe ratio (test months) | 0.54 | 0.78 | 0.64 |
| Sharpe ratio in training, average of the chosen lengths | 0.93 | ||
| Decay (1 minus test over training) | 42% | ||
| CAGR | 3.9% | 6.0% | 6.9% |
| Maximum drawdown | -22.4% | -14.4% | -30.1% |
| Growth of $10,000 | $20,321 | $29,246 | $33,834 |
Rolling walk-forward: the length from 2 to 18 months with the highest Sharpe ratio in each 120-month training window trades the next 36 months; the test windows are stitched. Risk-free rate 0; cash earns 0. Jun 2008 to Sep 2026.
| Fold | Training window | Test window | Chosen length | Training Sharpe | Test Sharpe | Test Sharpe at 10 | Test Sharpe, buy and hold |
|---|---|---|---|---|---|---|---|
| 1 | Jun 1998 to May 2008 (120) | Jun 2008 to May 2011 (36) | 10 | 0.94 | 0.66 | 0.66 | 0.29 |
| 2 | Jun 2001 to May 2011 (120) | Jun 2011 to May 2014 (36) | 16 | 1.18 | 0.41 | 0.78 | 0.86 |
| 3 | Jun 2004 to May 2014 (120) | Jun 2014 to May 2017 (36) | 8 | 1.09 | 0.60 | 0.77 | 0.69 |
| 4 | Jun 2007 to May 2017 (120) | Jun 2017 to May 2020 (36) | 8 | 0.72 | 0.52 | 0.06 | 0.51 |
| 5 | Jun 2010 to May 2020 (120) | Jun 2020 to May 2023 (36) | 2 | 0.87 | -0.15 | 1.02 | 0.47 |
| 6 | Jun 2013 to May 2023 (120) | Jun 2023 to Sep 2026 (40) | 8 | 0.78 | 1.26 | 1.44 | 1.46 |
Months in brackets. The chosen length is the one with the highest Sharpe ratio in the training window (the shortest on a tie). Data from Jun 1998, 340 months. The last test window runs to the end of the data. Jun 2008 to Sep 2026.
How MarketHeist computes it
Every choice an implementation has to make, as the code makes it today. The page is checked against the code each time it is built.
| Choice | How it is computed | In the code |
|---|---|---|
| Data | Monthly total returns (dividends reinvested). Funds younger than the period shown are extended backward with older funds, spliced on returns, not prices. | |
| Window | The test months of the worked example and the lab are the stitched test windows, Jun 2008 to Sep 2026 (220 months). The folds are cut from the months after the longest filter length the lab offers (24 months) has its first signal; the last test window runs to the end of the data: months left after the last full window form a fold of their own when there are at least 12 of them, and otherwise are added to the previous test window, so no month is dropped. | |
| Portfolio streams | A fund is its proxy-extended monthly return stream. A built-in portfolio is read as the MarketHeist MCP reads it: its composed stream with its own rebalancing rule and overlay. A custom mix is rebalanced yearly. | |
| Walk-forward | In each fold, every filter length in the lab's range is scored by its Sharpe ratio over the training window and the highest-scoring length (the shortest on a tie) trades the next test window; the test windows are stitched into one series. A rolling fold slides the training window forward by one test window; an anchored fold keeps its start and grows. The comparison filter has a fixed length of 10 months, and buy and hold is measured over the same test months. Decay is 1 minus the stitched test Sharpe ratio over the average training Sharpe ratio of the chosen lengths. | |
| What is chosen | The length N of the whole-portfolio trend filter (hold while the month-end value is above its N-month average, otherwise cash at 0, applied the next month, no costs). Each length's filtered returns are computed once on the full history, so every length is warmed up when a fold opens. | walkforwardDatacontent-engine/lab/compute4.ts |
| Window and folds | The lab window starts after a 24-month warm-up. Folds start at its first month and step forward by one test window; only complete test windows count, so the latest months that do not fill one are left out until enough data arrive. Rolling training windows keep their length; anchored ones start at the window's first month and grow. | walkFoldscontent-engine/lab/compute4.ts |
| Choice and scores | In each training window, the length with the highest Sharpe ratio (monthly, risk-free rate 0, ×√12; the shorter length on a tie). In-sample = the average of the chosen lengths' training Sharpe ratios; out-of-sample = the Sharpe ratio of the joined test returns; decay = 1 − out-of-sample / in-sample. Switching length between folds costs nothing. | |
| Fixed comparison | The 10-month length, the setting chosen in advance, and buy-and-hold, both over exactly the joined test months. | WALK_FIXED_Ncontent-engine/lab/compute4.ts |
| The backtester's walk-forward | Runs on the backtest's own bars (usually daily) with in-sample, out-of-sample and step sizes in bars and a grid of indicator parameters, scored by Sharpe ratio, CAGR, Calmar ratio or maximum drawdown. Windows are anchored to the end of the data, so the last test window ends on the latest bar and leftover bars at the start serve only as warm-up. Each parameter set is run once on the full history and sliced, so indicators are warm; each window starts in cash and enters on its first bar, paying the entry cost; a set that never holds a position in the training window cannot win it. It reports the mean in-sample score, the mean and median out-of-sample scores, degradation as their difference, the share of positive test windows and the chained out-of-sample equity curve. A validate mode runs fixed parameters through the same windows. | runWalkForwardapps/backtest/src/engine/walkforward.ts |
| Backtester windows | Windows step back from the last bar by the step size; complete windows only. | wfWindowspackages/shared/engine/robustness.ts |
| Engine functions | The functions the portfolio app and the MarketHeist MCP tools run, called here on the same monthly returns. | applyPortfolioOverlaypackages/portfolio/engine/overlay.tssharpepackages/shared/engine/metrics.tsmetricsFromReturnspackages/portfolio/engine/portfolio.ts |
Walk-forward analysis compared with its neighbours
| This lab (portfolio, monthly) | Backtester (strategy, its bars) | |
|---|---|---|
| What is chosen | Trend-filter length for a whole portfolio | Any grid of indicator parameters |
| Data | Monthly portfolio values | Daily, weekly or monthly bars of one ticker |
| Score | Sharpe ratio | Sharpe ratio, CAGR, Calmar ratio or maximum drawdown |
| Window placement | From the first month after warm-up; latest partial test window left out | From the last bar backwards; earliest leftover bars are warm-up |
| Window start | Continues the filter's state; no switching cost | Starts in cash; enters on the first bar and pays the entry cost |
| Degradation | 1 − out-of-sample Sharpe of the joined record / average in-sample Sharpe | Mean in-sample score − mean out-of-sample score |
| Comparison shown | Fixed 10-month length and buy-and-hold over the same test months | In-sample vs out-of-sample per window; share of positive windows |
Limitations
- Few independent test windows. With multi-year test windows a long history holds only a handful, and one good or bad window can decide the result.
- The analysis has its own settings: training and test lengths, rolling or anchored windows, the score and the candidate range. Trying many combinations and reporting the most flattering one repeats the overfitting it is meant to catch.
- Training windows overlap heavily when they roll by a short step, so consecutive choices are not independent decisions.
- The lab's version ignores trading costs and taxes, and cash earns 0; a filter that switches length or position often would lose more to costs than shown.
- Long histories are extended backward with older funds or indexes where a fund is younger than the window, so early folds approximate the funds rather than reproduce them.
Frequently asked questions
- What is the difference between walk-forward analysis and a train/test split?
- A train/test split optimizes once on an early period and tests once on a later one. Walk-forward analysis repeats that step many times, moving forward through the data, so the out-of-sample record covers most of the history and the choice is re-made as new data arrive, as it would be in practice.
- How much in-sample to out-of-sample decay is acceptable?
- There is no universal threshold. Some decay is expected, because the in-sample winner is partly the luckiest setting. A more useful test is whether the out-of-sample record beats a fixed setting chosen in advance and a buy-and-hold alternative over the same months; if not, the optimization added nothing.
- Rolling or anchored training windows: which one to use?
- It depends on whether old data stay relevant. Rolling windows adapt to change but use less data and switch settings more; anchored windows are steadier but slow to react. Choose one before looking at the results; trying both and keeping the better one is itself a selection.
- Does a good walk-forward result prove a strategy works?
- No. It shows that the selection rule held up on one historical path with the windows chosen. It is stronger evidence than an optimized backtest, but the test windows are few, they come from the same history, and the analysis has settings of its own.
References
- Pardo, R. (1992) Design, Testing, and Optimization of Trading Systems. Wiley. The walk-forward procedure for trading systems.
- Pardo, R. (2008) The Evaluation and Optimization of Trading Strategies. Wiley, second edition. Walk-forward analysis and walk-forward efficiency.
- Bailey, D. H., Borwein, J. M., López de Prado, M. and Zhu, Q. J. (2014) Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society 61(5). Why in-sample performance decays out of sample.
- López de Prado, M. (2018) Advances in Financial Machine Learning. Wiley. Limits of walk-forward testing and combinatorial purged cross-validation.
Try it on your own holdings
Run a walk-forward optimization in the backtester →
Or ask an assistant connected to the MarketHeist MCP server (https://marketheist.io/mcp), which computes the same figures:
Using MarketHeist.io, run a walk-forward test of a moving-average trend filter on the Bogleheads three-fund portfolio: pick the length from 2 to 18 months with the highest Sharpe ratio on each 10-year window, test it on the next 3 years, and compare the out-of-sample Sharpe ratio with a fixed 10-month filter.
Ask an AI assistant
The prompt opens in a new tab, already filled in. An assistant without the MarketHeist connector answers from its own knowledge, not from the figures on this page.
Historical simulation, not advice. Computed by the MarketHeist portfolio engine from monthly total returns, data through Sep 2026. Markdown version of this page: walk-forward-analysis.md.