There was no plan.
Historians in the future will find no evidence of a conscious or coordinated effort. The path that led to an environment so perfectly suited to one unique kind of investment strategy changed the very nature of how people thought about the enterprise itself. Yet that path, which we still appear to be on, was completely unintentional.
Active professionals had been applying increasing amounts of intelligence, money, experience, information and time to maintain their investment edge. Whatever was driving their actions, it didn't have the goal of creating an ecosystem so antithetical to their interests that it would endanger their own existence. And yet, that is exactly what happened.
In 1993, I came across the then-recently published book, "Investment Policy" by Charles D. Ellis. Among other insights, Ellis explained why professional investors had (and would continue to have) difficulty producing above market returns. He laid out his thinking so clearly and compellingly that I refer to it even now, and he revisited the same theme earlier this year in a piece for the Financial Times, "The End of Active Investing?"
Ellis's argument is built on the observation that as professional investors began to dominate the market, it became more difficult to produce market-beating returns. Today, there are 135,000 chartered financial analysts (or CFAs), another 200,000 studying for the exam and an estimated 320,000 Bloomberg terminals providing over 1 million professionals with sophisticated access to anything and everything related to investing — in short, all the makings for an amazingly efficient market.
Ellis identified four distinct phases over the last 50 years that led to today's market environment: In the first phase, institutional market share was less than 20%, and active managers easily outperformed the averages. In the second phase, institutional market share was less than 50%, and active managers generally equaled market returns (before expenses).
In the third phase, institutional market share began to dominate the market, and active managers regularly underperformed. In the fourth phase, as institutional market share rose above 90%, so few active managers matched market returns that investors, large and small, started to seriously question the value of any active management.
It gets worse (for active investors, that is). John Bogle, who got the indexing ball rolling in 1975, says institutional market dominance is not relevant to indexing's advantage over active alternatives and that markets were efficient even when institutions were small participants.
Writing in the Financial Analysts Journal last year, Bogle compared the returns of the S&P 500 index to average equity mutual fund returns in two 30-year periods: 1945-1975 and 1985-2015. Even though institutions were an inconsequential part of the market in the first period and dominated it in the second one, the index beat the funds in each timeframe by 1.6% per year.
The Market Wins
Whether you agree with Ellis or Bogle, the only logical conclusion you can make is that it is a fool's errand to think you can beat the market, and it is a fool's errand to think you can pick an active manager who can beat the market.
It doesn't matter whether individual investors are making the market efficient or institutions are making the market efficient. The market is going to be efficient enough to give indexing what it needs to maintain an advantage over almost anyone who tries to do it themselves.
Oh, a few people will succeed: Warren Buffett, Klarman and some others perhaps. There are meaningful studies that show that value and momentum have persistence, and over many years can produce above-average returns. But the odds of identifying the managers who can successfully deliver those returns are not great.
The typical criticisms of indexing have always felt slightly inadequate. Consider just a few: The top heaviness of indexes makes them dangerous during bubbles. We've had two massive market bubbles and crashes since 2000 and indexes have come out looking just fine.
One recent blogger suggested that the last trillion dollars coming into index funds was "hot money" — i.e., investors chasing performance — and this hot money would rush out at the first sign of trouble, leading to a market collapse. That may be so, but I doubt any meaningful amount of money coming out of index funds will be flowing back to active managers any time soon. That ship has sailed.