With much fanfare, Morningstar introduced a new fund grading system in November, and so far the results have not been inspiring. During the first quarter of this year, top-rated large-cap blend and mid-cap value funds trailed category averages by half a percentage point. Top-rated small-cap blend funds lagged by 2 percentage points. While it is too soon to draw firm conclusions about the new system, the early returns should serve as a warning for advisors that Morningstar’s recommendations may sometimes miss the mark.

Under the new system, recommended funds can win rankings of gold, silver or bronze. Many advisors may be inclined to ignore the whole thing, but that will be hard to do. Funds can advertise that they have won gold, and soon clients will begin asking why they should hold anything that is rated only silver or bronze.

Though the new awards are likely to become the most prominent grades issued by Morningstar, the old star system is not being eliminated. So now funds will carry two grades. Many top funds will be rated as gold and be awarded 5 stars, the highest ranking under the old system. But the two ratings systems do not always agree. In some instances, 2-star funds are rated gold. Such mixed grades are bound to leave advisors and clients scratching their heads.

The grades can vary because the two Morningstar systems use different criteria. The stars are based strictly on past performance. Funds receive five stars when they land in the top 10 percent of their categories for risk-adjusted returns. As many academic studies have shown, the star system is not necessarily a reliable guide. Past performance does not guarantee future results, and many five-star funds have failed to deliver strong returns. The stars can be particularly unreliable because the system is purely mechanical. Say a longtime value manager won five stars and then quit. The replacement may be a growth manager with little experience. The fund will still carry five stars, even though there could be no connection between the future performance and the past results.

Instead of being based solely on past performance, the new ratings rely on the judgment of analysts about the future outlook. Morningstar awards a gold grade to the funds that seem most likely to outperform their categories over the next five years. The analysts favor experienced managements that follow disciplined strategies. Funds get high marks for charging low fees. Past performance counts—but it is only one of a number of factors. “We are trying to identify funds with the kind of strong competitive advantages that will enable them to deliver good risk-adjusted performance,” says Russel Kinnel, Morningstar’s director of mutual fund research.

Morningstar emphasizes that its two grading systems serve different purposes. The stars are intended to provide a convenient summary of past performance—not a prediction of the future. The new grades are forward-looking recommendations. While Morningstar’s reputation will rest on the performance of its new recommendations, the company says that advisors should do their own research and not rely on the ratings blindly. Some advisors agree that the new ratings can be helpful. “If you look at both the stars and the analyst ratings, then you will have a lot of useful information,” says Lewis Altfest, CEO of Altfest Personal Wealth Management, a financial advisor in New York.

Can the forward-looking system actually work? Perhaps. Morningstar already has a sound track record for making forward-looking recommendations. Since 1999, the company has been selecting a list of analyst picks, which includes about 170 funds. The performance of the picks list is worth considering because it uses the same criteria as the new forward-looking ratings. After being selected for the picks list, most funds went on to outperform their categories during the next five years. Of the U.S. stock funds on the list, 70 percent surpassed their categories. Bond funds did even better. Among taxable bond funds on the list, 89 percent succeeded. The picks list will now disappear, replaced by the new ratings.

So far many of the new rankings are unsurprising. Templeton Global Bond (Ticker: TPINX), the top-performing member of its category during the past five years, won a gold rating. Other standout funds with gold rankings include PIMCO Total Return (Ticker: PTTRX), T. Rowe Price New America Growth (Ticker: PRWAX), and Vanguard Wellesley Income (Ticker: VWINX). But there are some controversial choices. Selected American Shares (Ticker: SLADX), a two-star fund that trailed 73 percent of its peers in the last five years, was awarded a gold rating. Morningstar concedes that the fund has made some mistakes. But the analysts argue that Selected American has experienced managers, low expenses, and a strong process for finding undervalued stocks. Another controversial gold choice may be Dodge & Cox Balanced (Ticker: DODBX), a two-star fund that has trailed 92 percent of peers in the moderate allocation category during the last five years.

In instances when the analysts aren’t sure about the outlook, they award a neutral rating. In that category is Columbia Value & Structuring (Ticker: EVRAX). The fund has a strong track record, but the longtime manager is leaving. Funds with glaring problems are given a negative rating. Templeton Developing Markets (Ticker: TEDMX) got the thumbs down partly because of its above-average expense ratio.

By the end of 2012, Morningstar expects to have analyst ratings for 1,500 funds. About 500 will come with 7-page reports. The new reports provide more detail than Morningstar has ever offered. Each report evaluates a fund based on five separate subjects, including fund costs, the quality of the portfolio manager, and the strength of the parent company. In order to win a gold rating, a fund must receive positive scores in all five areas. Artisan International (Ticker: ARTIX) won a silver rating. Morningstar gave the fund’s investment process a passing grade, but knocked off points for having an above-average expense ratio. Advisors may disagree with the final ratings, but the reports should still be useful because they provide plenty of data and carefully reasoned evaluations.