Here is Wikipedia’s definition of Survivorship bias; “ in quantitative backtesting of market performance or other characteristics, survivorship bias is the use of a current index membership set rather than using the actual constituent changes over time. Consider a backtest to 1990 to find the total return of S&P 500 members who have paid dividends within the previous year. To use the current 500 members only and create a historical equity line of the total return of the companies that met the criteria, would be adding survivorship bias to the results. S&P maintains an index of only healthy companies, removing companies that no longer meet their criteria as a representative of the large-cap U.S. stock market.”
Backtesting an algorithm will suffer from survivorship bias if historical data does not include delisted stocks.
Imagine an extreme case, a model buys the one stock that dropped the most in the previous day and holds it until it makes a profit. This is a terrible strategy but we’re trying to make a point. Testing would only show symbols that were beaten down but subsequently survived. What has not been considered is all the stocks the model bought but were subsequently delisted, or in other words, fell to zero. This model would have never tested those symbols because they were not in the database.
If testing a trading idea where the universe (or market) will be a particular index, e.g. Nasdaq 100 because the members of this particular index are large and have a lot of liquidity, and its known backtests have to be run over an extended period of time in order to ensure the system has been through different kind of markets (bull, bear, side, low/high volatility, etc.), if the model doesn’t take into account survivorship bias it effectively has tested a strategy against a VERY different environment than the one that existed in reality. Case in point, only 60 of NASDAQ members in 2010 were part of the index in 2005. To say it differently, 40 stocks were dismissed because of poor stock performance, bankruptcy or acquisition.
If trading US equities and backtesting more than 6 months survivorship bias has to be considered. If not, your model will be trading with erroneous data.
Check our out the Performance Matrix for our algorithms and sign up today.