Skip to content
Zerotoinvest
DEEN

Backtesting

In backtesting you apply rules to old price data and see what would have come out. The test is only as good as the care you take to avoid the typical mistakes.

Learning objective: After this lesson you can check a backtest result for typical mistakes instead of believing it blindly.

1 min read Last checked: 2026-09-24

The idea sounds tempting. You take your rules, run them over the last ten years and see what would have come out. In practice almost every backtest is too optimistic.

Common Mistakes

The most common mistakes are always the same. You use data that didn't exist yet at the time of the decision. You only test with companies that still exist today and overlook the ones that went bust. Or you forget fees, the spread and taxes.

The Mistake You Make Yourself

The sneakiest mistake is you. You already know the past. If your first test looks bad and you tweak the settings until it looks good, you haven't tested anything. You've fitted the rule to the past.

How to Test More Cleanly

In practice that means: fix the rules completely before you look at the data. Test on one period and check the result on another that you haven't touched before. And be generous with costs, not stingy.

Summary

  • Almost every backtest is too optimistic.
  • Fix the rules before you look at the data.
  • A test period is no longer independent once you use it several times.

Did you get it?

What is survivorship bias in a backtest?

The data set only contains companies that still exist today. The failed ones are missing.

Why are figures from annual reports a problem?

Because they get corrected afterwards and today's databases don't show what was known at the time.

Why isn't a test period split off once enough?

Because it loses its independence as soon as you use it several times and change your rules in between.

What others often ask about this

Why does my strategy work in the backtest but not live?

Usually it's fitted too closely to the old data. Add costs, spread and slippage that are missing from the test, and sometimes a mistake where the test uses information that wasn't available at the time.

How do I test more honestly?

Test on data you didn't look at while developing, include realistic costs and be suspicious of very smooth results.

Check your understanding

Your task

Work out how many dollars 100 trades of $5,000 each cost per year at 0.2 percent per trade. Then decide how many years of your data you will leave untouched as a validation period before you run the test once, and write down your rules completely before you look at the data.

Sources and further reading

  • The widely cited paper on the probability of backtest overfitting, showing why standard statistical safeguards against overfitting are unreliable for trading strategies. View source ↗

Related

Where to go from here

Next lessonOverfitting