On the Limitations of Backtesting
You might have noticed, especially if you are getting into trading, that traders are confronted with two bits of crucial advice that seem somewhat contradictory.
On the one hand, brokers will insist in all their disclaimers that “past performance does not predict the future result.”
On the other, there will be more experienced traders who will insist that backtesting your results, and finding out your relative risk in trading are vital aspects.
How is backtesting useful if past performance doesn’t predict future results?
About future results
The future is, by definition, uncertain – and the admonishment that past performance doesn’t predict future results is rather strong. This is because brokers and even other traders giving advice don’t want to be accused of doing the impossible: predicting the future.
If anyone could predict the future, there wouldn’t be any point in having markets. And the purpose of trading and developing strategies is to try and narrow down the possibilities.
So, the idea that you can’t rely on the past to define the future of the markets has some solid basis.
On the other hand, although unexpected events do occur, there is a certain amount of predictability to the markets based on underlying fundamentals and trading psychology.
For example, recessions. They happen; there will be one in the future – barring something that we can’t imagine now (hence, the disclaimer).
We cannot, however, know with certainty when the next one will happen, or how it will develop. Building up reserves in times of economic growth then becomes a reasonable strategy. This is because even though you can’t predict when a recession will happen, you can know that one will happen, and be ready.
So backtesting, then…
Backtesting allows you to try out your strategy with no risk, given the prior performance in the markets. This is helpful to understand how it performs in the real world. Particularly since the past has already happened, so you know for sure how the market performed.
Backtesting does not account for unexpected market swings due to unforeseen events; which is the caveat that gets included in all market advice.
The markets perform relatively within expectations for the simple reason that traders are acting out of reasonable self-interest in response to the data that is available.
This is what makes trading an investment and not gambling. Indeed, inflation data is not random. For example, intervention from a central bank to either support inflation to reach its target or to pair back an overheated economy is not accidental.
A strategy, therefore, can be developed within the relative amount of predictability that the market offers. But we also have to take into account the relative amount of uncertainty that is ever-present in the market.
Backtesting allows you to review how a given strategy will perform in certain market conditions. This enables you to a certain level of predictability in how it will perform, should those conditions repeat. Then, you have had to factor in the possibility of unexpected events (the increased relative risk) to account for the fact that performance does not guarantee future results. Then incorporate security measures, such as stop losses, lot size, and holding reserves.
When backtesting, you also have to remember that you are not subject to the psychological strain of trading; with having actual money in the market, your decisions might be different than when backtesting, similar to the difference between trading a demo account and a funded account.