Last week, we published Morningstar’s response to our article, Morningstar Ratings Fail over a Full Market Cycle, which we had published the previous week. Here is our response to Morningstar’s:
I am not persuaded by Morningstar’s objections to our study. I will respond to the three points raised by John Rekenthaler, Morningstar’s VP of Research.
Rekenthaler claims that three years, the time period we chose for our evaluation, is too short to demonstrate the success or failure of Morningstar’s rating system. I agree with Rekenthaler that studying a longer time period would provide more informative results, but as I note below in response to his third objection, using a longer time period introduces substantial survivorship bias.
I believe three years is enough time, especially when one can compare the relatively stable bull-market conditions of 2004-2006 to the turbulent conditions of the three years ending 9/30/09. On that count, the evidence is incontrovertible. The ratings lost virtually all their predictive power once conditions became turbulent.
I chose three years as the measurement period because that was the time period used by Morey and Gottesman in their study, which is the most widely-cited examination of Morningstar’s new ratings methodology.
My choice of three years is supported by the research firm Dalbar, which calculated that the average holding period for mutual funds in 2008 was 3.89 years. Calculating the average holding period, however, is not straightforward and Avi Nachmany, the Research Director at the research firm Strategic Insight, told me that fund companies often use five to six years as a representative expected holding period.
If Morningstar truly believes that their ratings are predictive only over periods as long as seven years, as Rekenthaler states in his response, then they should make that clear in their licensing. Fund companies should be required to include a disclaimer in their marketing literature: “Funds must be held at least seven years for this rating to be meaningful.”
Rekenthaler’s second objection is that the three-year time period I chose does not constitute a “full market cycle,” as I claim it did. I strongly disagree. The three years ending 9/30/09 include the bear market of 2008 and the bull market of 2009. That is as complete a market cycle as one could require for this type of study. The fact that the S&P 500 began the period at 1055 and ended at 1355 is less important than the inclusion of bull and bear market results.
The third objection raised by Rekenthaler is, on the surface, the most valid. He presents data indicating that the ratings had predictive power over the seven years ending 6/09. If this were true, it would be a strong endorsement of the ratings.
Very few subjects have commanded more academic attention than the persistence of mutual fund performance. While some evidence has been found that future performance can be predicted solely based on past performance, those findings typically hold for periods less than two years (e.g., using a “hot hands” approach). No scientific evidence exists that performance can be predictive over periods as long as seven years. Morningstar’s findings, if they could pass academic scrutiny, would be a breakthrough.
