Bias Controls & Validation

What it is

Look-ahead bias occurs when a strategy uses information that would not have been available at the time a trading decision was made. The most common form is using the closing price of bar T to generate a signal, and then executing that trade also at bar T's close. In reality, the close price is only known after the bar closes — at which point the market is already closed.

Other forms of look-ahead bias we guard against

• Using bar T's high/low to trigger stops — these are only known after the bar closes
• Indicators computed with a "future" period (e.g., a centered moving average)
• Normalization using the full dataset's min/max rather than rolling min/max
• Strategy parameters optimized on the full period, then tested on the same period
• Earnings surprise signals using actual EPS before the announcement date

Survivorship bias occurs when a backtest only includes stocks that are currently listed on the exchange. Companies that went bankrupt, were acquired, or were delisted during the test period are excluded — even though they were valid trading candidates at the time. The result is that historical returns look much better than they actually were, because all the "losers" have been silently removed from the dataset.

Overfitting occurs when a strategy has been optimized so heavily on historical data that it has effectively "memorized" the past rather than learned a generalizable rule. A strategy with 12 parameters, all tuned on 3 years of data, will show spectacular historical results — and fail immediately in live trading.

How we detect and prevent overfitting

Selection bias occurs when the choice of which instruments to backtest is influenced — consciously or not — by knowledge of how those instruments performed historically. Testing a momentum strategy only on the biggest winners of the last decade will produce a biased result.

Optimization bias is a specific form of overfitting. When you run a grid search over hundreds of parameter combinations and pick the best performer, you are not finding the "true" optimal parameters — you are finding the parameters that happened to work best on that specific historical period due to random noise.

Our optimization rules

• ● Strategy parameters are set before running the in-sample backtest
• ● If optimization is used, it is performed only on the training set — never on the validation set
• ● We report the optimization surface (not just the peak) so readers can see robustness
• ● We prefer parameter values that lie on flat plateaus, not narrow peaks
• ● Walk-forward optimization: parameters are re-optimized periodically, simulating real adaptive management

Out-of-sample (OOS) testing is the most important validation step. After a strategy is developed and optimized on the training set, it is tested once on a completely separate holdout period. The OOS result cannot be used to further refine the strategy — if it is, the holdout is no longer truly out-of-sample.

Walk-Forward Testing

Walk-forward testing is a more sophisticated form of OOS testing that simulates how a real trader would periodically re-optimize their strategy. Instead of one fixed training/test split, we use a rolling window: optimize on the first N bars, test on the next M bars, advance by M bars, and repeat.