Morningstar Slams Study Questioning Its Rating System

"First, we're skeptical of any study that finds that fund expenses have no predictive value," Morningstar said. "Expenses have been shown time and again to be predictive of performance. We think it's also important to point out that Gottesman and Morey did find that a high culture score was strongly associated with low fees, long manager tenure, low turnover and fund survival. These are all very positive qualities in a mutual fund."

Second, the company says the Gottesman-Morey study only examined domestic equity funds, which represent about half of Morningstar's stewardship grade coverage, over a single five-year time period.

"While we generally wouldn't recommend drawing conclusions from such a short period, it's worth pointing out that the study did find predictive power over that full five-year period, just not the shorter periods," Morningstar said. "Given the long-term nature of stewardship this makes sense. We wouldn't necessarily expect the predictive power of corporate culture to show up in the one- and three-year periods because culture affects things such as how good the next manager is when the current one departs, long-term fee policies (see Vanguard Admiral), not allowing a fund to get too big, etc."

Third, Gottesman-Morey's method for addressing survivorship bias actually inflates the performance of funds with poor culture grades, Morningstar says. They replace dead funds' track records after the death date with the performance of other funds with the same category, culture grade, expenses and turnover, but not the same performance history.

"I think it's safe to say that most funds that were merged or liquidated in the periods they study did so because of poor performance," the Morningstar statement said. "As a result, any surviving funds in the lowest culture group would have had much better performance than the ones that didn't survive. Additionally, our stewardship coverage is biased toward larger, better-performing funds to begin with, so selecting any fund with stewardship coverage biases performance upward."

The statement continued: "Fourth, one of Gottesman-Morey's primary measures of 'success' is the Sharpe ratio, which does not take a fund's category into account. This will muddle the results. For example, the differences in Sharpe ratio between large value funds and small growth funds are so big that they overshadow the differences between A Stewards and C Stewards.

"Finally, we conducted our own study last year that found that culture grades are predictive of fund performance. We examined how funds have performed since Morningstar first issued its stewardship grades in 2004 and again after we revised our Stewardship methodology in 2007. We concluded that funds with high stewardship grades (those receiving grades of 'A' or 'B') are very likely to survive in the long term, and more likely to provide competitive risk-adjusted returns in the ensuing period."