The Underlying Difficulty in Forecasting Equity Returns

Austin Murphy and Zeina AlSalman, authors of the study, “The Continued Forecasting Effectiveness of a Real Earnings Model of the Equity Premium,” published in the July 2023 issue of The Journal of Investing, examined the empirical performance of the real-earnings model in forecasting annual excess returns on the stock market. They began by noting an important advantage of the real-earnings model: “In contrast to alternative models of the equity premium presented in leading academic journals that require statistical estimation of parameters (and have been found to exhibit poor forecasting performance after publication), this model estimator has theoretical foundations that do not require estimation of any parameters.” They added that the real-earnings yield theory of stock returns is consistent with the empirical findings of profits rising with inflation over the long term. The real-earnings model adds the E/P ratio (which provides an estimate of the real return) to the difference between the yield on the nominal 10-year Treasury note and the yield on 10-year TIPS (providing an estimate of inflation). Together they provide an estimate of the nominal expected return to stocks.

Murphy and AlSalmon noted that prior research using the formula (taking the highest earnings of the S&P 500 over the prior 10 years to calculate the E/P) had found a correlation between the equity premium and the subsequent annual excess stock returns (over the following five-year period) on the S&P 500 of over 0.30. That study covered the period 1998-2010. Their new study, covering the period through August 2021, found that after the 2010 end point of the original tests, the correlation between the model and subsequent annual excess stock returns was an even higher 0.38. They also found that subsequent excess returns on the stock market indicated an even higher correlation of 0.52 over the 2015-2021 period. Over the entire 2003-2021 interval, the correlation between the model forecast and the subsequent annual excess return on the S&P 500 was 0.41. They added: “An analysis of the longer-term forecasting power of the model, the correlation between the model equity premium and the annual compounded excess return on the S&P 500 over the subsequent five years (that included the model forecasts from 2003–2017) is found to be 0.84.” Given the historical volatility of stock returns and the difficulty in forecasting stock prices in the short term, it’s not surprising that the model’s ability to forecast returns improved as the horizon extended.

Their findings led Murphy and AlSalmon to conclude: “The correlation between these model estimates and the following year’s excess returns on the stock market is found to be far higher than what has been reported after publication for the many alternative models of the equity premium that have been presented in the finance literature.”

How to treat estimated equity returns

As Damodaran noted, the equity risk premium can change greatly and quickly. It’s critical that investors treat any forecast only as the mean of a wide possible dispersion of returns. For example, consider the results from Cliff Asness’ study on the Shiller CAPE 10’s ability to forecast future returns.

In a November 2012 paper, “An Old Friend: The Stock Market’s Shiller P/E,” Asness, of AQR Capital Management, found that the Shiller CAPE 10 does provide valuable information. Specifically, he found 10-year-forward average real returns dropped nearly monotonically as starting Shiller P/Es increased. He also found that as the starting Shiller CAPE 10 ratio increased, worst cases became worse and best cases became weaker. Additionally, he found that while the metric provided valuable insights, there were still very wide dispersions of returns. For instance:

• When the CAPE 10 was below 9.6, 10-year-forward real returns averaged 10.3%. In relative terms, that is more than 50% above the historical average of 6.8% (9.8% nominal return less 3.0% inflation). The best 10-year-forward real return was 17.5%. The worst 10-year-forward real return was still a respectable 4.8%, just 2.0 percentage points below the average (29% in relative terms). The range between the best and worst outcomes was a 12.7 percentage point difference in real returns.

• When the CAPE 10 was between 15.7 and 17.3 (about its long-term average of 16.5), the 10-year-forward real return averaged 5.6%. The best and worst 10-year-forward real returns were 15.1% and 2.3%, respectively. The range between the best and worst outcomes was a 12.8 percentage point difference in real returns.

• When the CAPE 10 was between 21.1 and 25.1, the 10-year-forward real return averaged just 0.9%. The best 10-year-forward real return was still 8.3%, above the historical average of 6.8%. However, the worst 10-year-forward real return was now -4.4%. The range between the best and worst outcomes was a difference of 12.7 percentage points in real terms.

• When the CAPE 10 was above 25.1, the real return over the following 10 years averaged just 0.5% – virtually the same as the long-term real return on the risk-free benchmark, one-month Treasury bills. The best 10-year-forward real return was 6.3%, just 0.5 percentage points below the historical average. But the worst 10-year-forward real return was now -6.1%. The range between the best and worst outcomes was a difference of 12.4 percentage points in real terms.

What can we learn from the preceding data? Starting valuations clearly matter, and they matter a lot. Higher starting values mean that not only are future expected returns lower (and vice versa), but the best outcomes are lower and the worst outcomes worse. But a wide dispersion of potential outcomes, for which we must prepare when developing an investment plan, still exists – high starting valuations don’t necessarily result in poor outcomes.

It’s also why an investment plan should include a “plan B,” a contingency plan that lists the actions to take if financial assets were to drop below a predetermined level. Actions might include remaining in or returning to the workforce, reducing current spending, reducing the financial goal, selling a home and/or moving to a location with a lower cost of living.

Investor takeaways

The evidence makes clear that estimating future equity returns isn’t a simple task. It’s why trying to time markets based on short-term forecasts has proven to so difficult, and why legendary investors such as Warren Buffett (“The stock market serves as a relocation center at which money is moved from the active to the patient”) and Peter Lynch (“Far more money has been lost by investors trying to anticipate corrections than has been lost in all the corrections combined”) advised against trying to time markets.

Because we must develop financial plans without the benefit of a clear crystal ball, we should use the best tools available. But when using these tools, the evidence demonstrates that we should have a healthy skepticism as to the accuracy of forecasts. We must be careful not to treat outcomes from models in a “deterministic” fashion. Instead, we should treat them only as the mean of a very wide potential dispersion of possible outcomes. As an example, I’m not aware of anyone who in 1990 predicted that through 2022, Japanese large-cap stocks would produce a return of just 0.2% per annum over the 33-year period.

Your comprehensive investment plan should include options you will exercise if the equity-risk premium turns out to be less than expected. You need to list actions you will take to prevent your plan from failing to meet its primary objective – that your assets outlive you.

Larry Swedroe is head of financial and economic research for Buckingham Wealth Partners.

All opinions expressed are solely his opinions and do not reflect the opinions of Buckingham Wealth Partners or its affiliates. This information is provided for general information purposes only and should not be construed as financial, tax or legal advice. Certain information is based on third party data and may become outdated or otherwise superseded without notice. Third party information is deemed to be reliable, but its accuracy and completeness cannot be guaranteed. By clicking on any of the links above, you acknowledge that they are solely for your convenience, and do not necessarily imply any affiliations, sponsorships, endorsements or representations whatsoever by us regarding third-party websites. We are not responsible for the content, availability or privacy policies of these sites, and shall not be responsible or liable for any information, opinions, advice, products or services available on or through them. Neither the Securities and Exchange Commission (SEC) nor any other federal or state agency have approved, determined the accuracy, or confirmed adequacy of this article. LSR-23-533.

Read more articles by Larry Swedroe