The Fox and the Hedgehog

One example of this problem can be seen in scientific research. A 2005 study by John Ioannidis aptly titled "Why Most Published Research Findings Are False" concluded, "The majority of hypotheses deemed to be true in journals in medicine and most other academic and scientific profession are, in fact, false," Silver wrote. Mind boggling, no? Yet, according to Silver, when Bayer Labs tested Ioannidis' findings, "they could not replicate about two-thirds of the positive findings claimed in medical journals."

Silver suggests an alternative system for probability analysis based on Bayes' Theorem (again, I'll spare you the details), which, in very simplified language, requires taking a hard look at our assumptions, which often are based on our biases. According to Silver, the advantage of the Bayesian approach is that it involves "an explicit acknowledgement that our prior beliefs and biases can affect how we interpret new evidence and what those prior beliefs are."

Silver cites the performance of the credit ratings agencies—primarily Moody's and Standard & Poor's—prior to the mortgage meltdown as an example of traditional statistical analysis based on very unrealistic assumptions. Before the crash, S&P determined that there was only a 0.12% (1 in 850) chance that the collateralized debt obligations it analyzed would fail within the following five years. In fact, 28% of them failed—an error of more than 200%. According to Silver and others, the problem was that both agencies based their analyses on two assumptions: that real estate defaults occur independently (when someone loses their job, for instance), rather than collectively (say, when the economy crashes) and that if the real estate bubble burst (which they anticipated), it would have a minimal effect. In fact, when a real estate crash appeared imminent, Moody's "increased the default probability in its model by 50%." In reality, they overlooked the extent to which the collateralized mortgage obligation market was leveraged: The default rates were "200 times (20,000%) higher than their models predicted."

As Silver tells it, the problem is that our brains are wired to make sense out of the world around us by looking for patterns. By itself, this tendency can lead us to see correlations where none actually exist, which is why we need scientific experiments to test our hypotheses. This problem is compounded, however, by our current state of information overload, in response to which "we focus on those signals that tell a story about the world as we would like it to be, not how it really is. We ignore the risks that are hardest to measure, even when they pose the greatest threats to our well-being. We make approximations and assumptions about the world that are cruder than we realize. And we abhor uncertainty, even when it is an irreducible part of the problem we are trying to solve."

How can we manage our biases to make better predictions in the real world? Silver cites an Isaiah Berlin essay called "The Hedgehog and the Fox," which is based on the notion that "the fox knows many little things, but the hedgehog knows one big thing." The idea is that there are two kinds of experts: hedgehogs, who are "type-A personalities who believe in Big Ideas—in governing principles about the world that behave as though they were physical laws and undergird virtually every interaction in society"; and foxes, who are "scrappy creatures who believe in a plethora of ideas and in taking a multiple of approaches toward a problem. They tend to be more tolerant of nuance, uncertainty, complexity and dissenting opinion. If hedgehogs are hunters, […] foxes are gatherers."

As you might imagine, foxes turn out to be the better predictors. They are more cautious, rely more on observation than on theory and constantly challenge their ideas and predictions in light of new information or insights. Those traits make them adaptable to finding new approaches if their old one isn't working. Hedgehogs look for simple rules about how the world works and, once they identify them, are aggressive in their defense, rarely changing or even hedging them.

Silver suggests that we try to be more fox-like in our predictions, but acknowledges that in our sound-bite world, hedgehogs are often more popular, especially in the media. This, of course, is a problem for professionals such as financial advisors, who need to be right in their predictions, but at the same time, need to convince others of their expertise.

For advisors, perhaps the solution is to do their homework as a fox. Then, when talking with clients or perspective clients, be slightly more hedgehogish: state the limits of their advice (no one can be certain of the direction of the markets or the economy, etc.), and describe how they manage that uncertainty to give their clients the greatest chance of success based on what they know right now and how they've built in feedback loops to monitor the results and make appropriate adjustments. In that way, an advisor can be an expert at helping clients separate the signal from the noise.