Financial Planning magazine recently ran an article on their online site purportedly to demonstrate that actively managed mutual funds provide better performance than passively managed mutual funds, such as the popular index funds. Whenever I see such claims, the securities attorney in me comes out and I have to perform a forensic analysis of the evidence presented to analyze and verify both the evidence and the argument presented.

I immediately knew that there were serious issues with the study because (1) the analysis was based on the three-year annualized returns of the funds, and (2) the analysis was based purely on such returns and the funds’ expense ratios. A proper analysis of a mutual fund’s performance should be based on a time period that includes at least one down year in the market. This allows an investor to see how well an actively managed fund protected the fund’s investors since one of the touted benefits of active management is the ability of the fund’s management team to respond to such situations and better protect their investors.

Costs are an important issue in mutual funds investing, as they directly reduce a fund’s bottom line and, thus, an investor’s end return. Two critical costs for actively managed mutual funds are a fund’s annual expense ratio and its turnover, or trading, costs. As Burton Malkiel pointed out in his seminal work, “A Random Walk Down Wall Street,”

The two variables that do the best job in predicting future performance are expense ratios and turnover.

Since actively managed funds typically have significantly higher turnover than do passively managed mutual funds, it is not unusual to see an actively managed fund’s trading costs exceed the fund’s annual expense ratio. The failure of the financial planning study to factor in such trading costs essentially invalidates the entire study.

It should be noted that the failure to include such trading costs may be due to the fact that while mutual funds are required to disclose their annual expense ratios, they are not required to disclose their trading costs. Nevertheless, to totally ignore such costs is indefensible given their impact on the true cost and performance of a fund. At the very least, a fund’s reported turnover could be used as a proxy for the actual trading costs.

Another alternative used by many investment professional to calculate a fund’s trading cost is the methodology suggested by investment legend John Bogle. Bogle’s method is to double the fund’s stated turnover ratio and multiply the result by 0.60. As long as you consistently use the same methodology for all fund’s, the results are acceptable.

In addition to the previously mentioned shortcomings, the study suffered from from other issues, including

• five of the scenarios (#6, #8, #10, #13, and #20) involved funds charging front-end loads in excess of 5 percent. There is simply no reason to accept load funds when comparable, if not better, no-loads funds are available;
• one of the scenarios (#16) involved a fund using C shares with an annual 12b-1 fee of 1 percent. C shares are a red flag to regulators, as they are usually used by unethical financial advisers trying to collect the same 1 percent annual advisory fees charged by RIAs without registering as an RIA as required by law;
• four of the scenarios (#2, #8, ##15 and #19) involved funds without a five-year track record, raising questions about the reliability of their performance record;
• five of the scenarios (#2, #3, #5 #13 and #16) involved “active” funds with turnover ratios less than 25 percent, much lower than the average turnover ratio that indicates active management;
• four of the scenarios (#9, #12, #13, and #16) involved “passive” funds with turnover ratios in excess of 100 percent, significantly higher than the single digit turnover ratio normally associated with passively managed mutual funds; and
• one scenario (#19) where the turnover ratio for an “active” fund was essentially the same as the turnover ratio for the accompanying “passive fund, 33 percent to 26 percent, respectively.

Adding the turnover costs for each fund to the equation, using financial planning’s original combinations, active management won 9 of the 20 combinations using a three-year period of comparison. Over a five-year period, passive management won 11 of the sixteen scenarios.

The attorney in me wanted to see what the results would have been had financial planning used a benchmark as the passive element in each scenario. In this case, I used Vanguard’s Growth Index fund as my benchmark, as it is designated by Morningstar as a large cap growth fund.

Over the three-year period chosen by financial planning, the actively managed funds won 16 of the twenty scenarios. Over a five-period, the passively managed funds won 11 of the sixteen scenarios.

My takeaway is that the results clearly show why three-years is simply not a valid period for evaluating a mutual fund’s performance, especially when the three years fails to include a down market period. It should be noted that the five-year period I used included 2014-2012, the three-year period used by financial planning, as well as 2011, a down year of the stock market,  and 2010. The market’s poor performance in 2011 definitely had an impact on the noticeable difference in three and five-year performances.

If you exclude the scenarios that had legitimacy issues, there were only four scenarios that merited consideration (assuming one ignores the three-year analysis issue). Out of those four scenarios, active management and passive management each won two scenarios.

The article as a whole raised an issue that I have raised before regarding 401(k)/404(c) plans. Most 401(k) plans try to qualify as a 404(c) plan in order to shift the plan’s investment risks onto the plan’s participants. For that reason, ERISA Section 404(c) states that a plan must provide plan participants with “sufficient information to make an informed decision.”

Both the DOL and the courts have stated that Modern Portfolio Theory (MPT) is the standard for assessing a fiduciary’s prudence under ERISA. The cornerstone of  MPT is factoring in the correlation of returns among investment options in order to combine investments that behave differently under various economic scenarios, the hope being to avoid significant financial losses.

And yet, ERISA does not expressly require that such information be provided to plan participants, even though the plan’s fiduciaries presumably have such information in order to choose prudent investment options for the plan. Without such information, plan participants are not provided with an opportunity to design an effectively diversified plan account.

My fear is that too many people blindly accept these types of “best of’ their findings because they either do not care or are unwilling, or unable, to do the due diligence necessary to verify the claims made. A significant number of the people who call me tell me that they choose their investments based on either a “best of” list in some publication or Morningstar’s star system, even though Morningstar has publicly stated that their star system is not intended to be used in such a manner.

In considering “best of ” lists. investors would most likely be best served by following the warning of former President Ronald Reagan – “Trust, but verify.”