Survivorship Bias: Why Backtests Overstate Returns

Key Takeaways

• Survivorship bias occurs when a backtest only includes assets that survived to today, excluding every bankruptcy, delisting, and merger that happened during the tested period.
• CRSP US stock data shows annualized survivor-free returns running 1 to 2 percentage points below naive survivor-only returns over long windows; for small caps and emerging markets the gap is wider.
• A small-cap value strategy on the current Russell 2000 showed 14 percent CAGR and Sharpe 0.9 on survivor-only data; switching to a survivor-free universe dropped CAGR to 10 percent and Sharpe to 0.55.
• Investors using Yahoo Finance or Google Finance for long-horizon backtests are almost always working with survivor-biased data, since those sources do not preserve delisted tickers.

What It Is

A dataset is survivorship-biased if the set of entities it contains was selected based on outcomes that postdate the backtest period. The most common form is an equity database that starts from today's S&P 500 membership and back-fills each stock's price history. Every firm kicked out of the index during the backtest window is missing. Every firm merged, delisted, or bankrupt is missing. Only the ones that made it to the finish line are there.

Brown, Goetzmann, Ibbotson, and Ross formalised the problem in their 1992 paper Survivorship Bias in Performance Studies in the Review of Financial Studies. They showed analytically that conditioning a return sample on survival distorts the risk-return relationship and can create the illusion of predictability where none exists.

The Intuition

Imagine a horse race. You record the finishing times of every horse that finished and compute the average. You forgot about the ones that dropped out mid-race because their times are not in the file. The average is too fast. That is exactly what survivorship bias does to a backtest, except the dropouts are bankruptcies.

The uplift is not small. CRSP US stock data commonly shows annualised survivorship-free returns around 1 to 2 percentage points below the naive survivor-only version over long windows. For small caps and emerging markets, the gap is wider because failure rates are higher. For mutual funds, Morningstar has documented that roughly half of funds that existed at the start of a decade no longer existed at its end.

How It Works

Survivorship bias enters through the data definition step, before a single line of strategy code is written.

Equities. Most vendors sell two versions of their history. A current constituents file lists today's tickers and their back-adjusted prices. A historical constituents or delisted file preserves every ticker that ever existed, including the ones that went to zero or were acquired. Only the second kind is survivorship-free. The CRSP US Stock Database and the Compustat "research" tapes are standard sources. CRSP also publishes a dedicated Survivor-Bias-Free US Mutual Fund Database.

Mutual funds and hedge funds. Funds that close, merge, or liquidate are often removed from commercial databases. This creates a double bias: the disappeared funds were typically poor performers, and voluntary reporting means struggling funds sometimes stop reporting before they close. The combined effect on hedge fund indices can exceed 2 percentage points per year.

Indices over time. A backtest on "the Nasdaq 100 from 1995 to 2024" using today's membership is not a backtest of the Nasdaq 100. It is a backtest of the 100 firms that happened to be in the index in 2024, applied retroactively. The real Nasdaq 100 in 1995 contained dozens of names that subsequently vanished.

A simple check: count the number of distinct tickers in your dataset versus the number that existed historically. If the ratio is close to 1, the data is survivorship-biased.

Worked Example

Take a small-cap value strategy backtested from 2000 to 2024. You pull the Russell 2000 current membership and build a monthly portfolio that ranks the 2,000 names by price-to-book and buys the cheapest quintile.

In the naive run, you see a 14 percent CAGR and a Sharpe of 0.9. Before celebrating, you replace the dataset with a survivor-free version that includes every firm that was in the Russell 2000 at any point during the backtest, including those that delisted. The eligible universe is now closer to 6,500 historical tickers.

Cheap small-caps are disproportionately the ones that later went bankrupt. The survivor-free CAGR drops to 10 percent and Sharpe to 0.55. Maximum drawdown grows from 38 percent to 52 percent because the failures now sit in the portfolio. Neither number is wrong. The first one is a fairy tale. The second is the question a real PM has to answer.

Common Mistakes

1. Running S&P 500 or Nasdaq 100 backtests with current constituents. Because index membership selects for past performance, using today's members gives you a portfolio of pre-screened winners. Multi-decade backtests on this setup often overstate returns by 2 percentage points or more annually.

2. Ignoring mutual-fund and hedge-fund closures. Failed funds disappear from commercial databases. If you pull a "ten-year top funds" list to evaluate a manager-selection rule, you are looking at survivors only. The CRSP Survivor-Bias-Free US Mutual Fund Database exists precisely to fix this for academic work.

3. Trusting long history on free data platforms. Yahoo Finance, Google Finance, and similar retail sources generally do not preserve delisted tickers. A long history on those platforms is almost always survivor-biased. They are fine for charting a living stock and wrong for backtesting strategy selection.

4. Assuming survivorship is only a US large-cap problem. It is worse in small-cap, worst in emerging markets, and particularly severe in microcaps, biotech, and crypto. Anywhere base-rate failure is high, the gap between survivor-biased and survivor-free returns is wide.

5. Not adjusting for reverse survivorship. A firm that is acquired at a premium leaves the dataset in a positive way. Treating every delisting as zero understates returns for strategies that pick takeover candidates. Good datasets record the delisting price or acquirer compensation so the last return can be modelled correctly.

Frequently Asked Questions

Q: What is survivorship bias in simple terms? Survivorship bias means your backtest only includes companies that still exist today, so it silently excludes every firm that went bankrupt, was acquired, or was delisted during the test period. The returns it reports are the returns of winners, which is not what you would have earned in real time.

Q: How does survivorship bias affect investment decisions? It inflates historical returns, making strategies look more reliable than they are. An investor who builds a small-cap value strategy on survivor-biased data and then trades it live will consistently underperform the backtest, because real portfolios hold stocks that fail.

Q: What is a real-world example of survivorship bias in backtesting? A researcher builds a cheap small-cap portfolio on today's Russell 2000 membership going back to 2000. Cheap small-caps disproportionately include companies that later went bankrupt. The survivor-only CAGR comes in at 14 percent; the survivor-free version, which includes all historical members, falls to 10 percent with a maximum drawdown 14 percentage points deeper.

Q: How can investors get survivorship-free data? The CRSP US Stock Database and Compustat research tapes include delisted ticker histories and are standard in academic research. The CRSP Survivor-Bias-Free US Mutual Fund Database covers funds. For retail investors, Alpha Architect and similar providers publish guidance on handling delistings properly.

Q: How is survivorship bias different from look-ahead bias? Look-ahead bias uses data from the future on the decision date. Survivorship bias excludes failed assets from the historical universe entirely. Both inflate backtest returns, but survivorship operates through the data selection step before any rules are applied, while look-ahead operates through incorrect timestamps on the data itself.

Sources

1. Brown, S.J., Goetzmann, W.N., Ibbotson, R.G., Ross, S.A. (1992). "Survivorship Bias in Performance Studies." Review of Financial Studies, 5(4), 553-580. https://academic.oup.com/rfs/article-abstract/5/4/553/1590264
2. Brown, S.J., Goetzmann, W.N., Ibbotson, R.G., Ross, S.A. "Survivorship Bias in Performance Studies" (working paper PDF). https://terpconnect.umd.edu/~wermers/ftpsite/FAME/Brown_Goetzmann_Ibbotson_Ross.pdf
3. Center for Research in Security Prices. "CRSP Survivor-Bias-Free US Mutual Fund Database." https://www.crsp.org/research/crsp-survivor-bias-free-us-mutual-funds/
4. Alpha Architect. "Dealing with Delistings: A Critical Aspect for Stock-Selection Research." https://alphaarchitect.com/dealing-with-delistings-a-critical-aspect-for-stock-selection-research/
5. Carhart, M.M., Carpenter, J.N., Lynch, A.W., Musto, D.K. "Mutual Fund Survivorship." https://pages.stern.nyu.edu/~alynch/pdfs/rfs02cclm.pdf

Disclaimer

This article is educational content only and is not financial advice. Nothing here is a recommendation to buy, sell, or hold any security. Consult a licensed advisor before making investment decisions.