Major Flaw with Five Star Ratings

Posted on the 11 April 2013 by Mdelp

%%bloglink%%

The National Highway Traffic Safety Administration (NHTSA) awarded several vehicles included both the 2013 Ford Focus and the 2013 Ford Explorer “5 Stars” for frontal crash safety.

These two vehicles from the same manufacturer both received the same safety rating, yet they are very different in other areas such as fuel economy, braking, power, etc.

Similarly, Morningstar awarded several mutual funds “5 Stars” based on returns including Oppenheimer Developing Markets (symbol ODMAX) and Oppenheimer Limited Term California Muni Fund (symbol OLCAX) but like the Focus vs. the Explorer, just because two funds were both rated “5 Stars” doesn’t mean they are identical in all areas.

  Oppenheimer Developing Markets primarily invests in stocks of companies in developing economies and has experienced a wide range of returns over the last ten years from a low of negative 48.03% in 2008 to a high of 81.73% in 2009.

Oppenheimer Limited Term California Muni Fund primarily invests in short term bonds of cities, counties and agencies in California and had a much narrower range of returns over the last ten years from a low of negative 12.69% in 2008 to a high of 21.26% in 2009.

This is where the major flaw of “5 Star” ratings shows up: how much of a difference between two products is allowed before measuring which one is better becomes almost meaningless?

NHTSA categorizes passenger cars by how much they weigh with the first weight division starting at 1,500 pounds and a different weight division every 500 pounds. Then they crash the vehicles into a fixed barrier at 35 mph which is equivalent to a head-on collison between two similar vehicles each moving at 35 mph.

Morningstar categories funds by what they primarily invest in (large U.S. stocks, long-term government bonds, etc.) then compares the returns between these funds.

But what if a fund is placed in the wrong category or takes on less risk than another fund?

Vanguard California Intermediate Term Tax Exempt Bond Fund (symbol VCADX) which also was given a Morningstar “5 Star” rating has an average effective duration of 5.06 years, worst calender year return in the last ten years was negative 2.07% in 2008 and the best calender year return was 10.25% in 2011. As of 3-31-13 data, one year returns were 5.49%, three year average annual returns were 6.28% and five year average annual returns were 5.55%.

Compare to Oppenheimer Limited Term California Muni (symbol OLCAX) whose average duration is currently 3.41 years and as of 3-31-13 data had one year returns of 5.38%, three year average annual returns of 5.54% and five year average annual returns of 4.93%.

Vanguard had a better one, three and five year returns and lost dramatically less than  Oppenheimer yet they both received “5 Star” ratings because Vanguard Cal Intermediate Term is classifed by Morningstar as Muni California Intermediate and Oppenheimer Limited Term Cal is classifed by Morningstar as Muni Single State Short. Two different categories so two different winners.

Yes, I still look at “Star Ratings” when considering what vehicle or fund to buy but it diffinately not the only thing I consider.