Mastering the Game of Imperfect Forecasting

Improving the Odds

It's a grand conundrum for the world of finance — we desperately need to make forecasts in order to serve our clients but we are remarkably poor at doing so.

The key then, as Roche has argued, is that we should shun low probability forecasts. By contrast, "Superforecasting" points to (and laughs at) the general inaccuracy of financial pundits at CNBC, whose forecasts are low probability ones of the highest order. As Jon Stewart famously remarked, "If I'd only followed CNBC's advice, I'd have a million dollars today — provided I'd started with a hundred million dollars."

The central lessons of "Superforecasting" can be distilled into a handful of directives. Base predictions on data and logic, and try to eliminate personal bias. Working in teams helps. Keep track of records so that you know how accurate (or inaccurate) you (and others) are. Think in terms of probabilities and recognize that everything is uncertain. Unpack a question into its component parts, distinguishing between what is known and unknown, and scrutinizing your assumptions. Recognize that the further out the prediction is designed to go and the more specific it is, the less accurate it can be.

In other words, we need rigorous empiricism, probabilistic thinking, a recognition that absolute answers are extremely rare, regular reassessment, accountability and an avoidance of too much precision. Or, more fundamentally, we need more humility and more diversity among those contributing to decisions. We need to be concerned more with process and improving our processes than in outcomes, important though they are. "What you think is much less important than how you think," says Tetlock. Superforecasters regard their views "as hypotheses to be tested, not treasures to be guarded." As Tetlock told Jason Zweig of The Wall Street Journal, most people "are too quick to make up their minds and too slow to change them."

Most importantly, perhaps, Tetlock encourages us to hunt and to keep hunting for evidence and reasons that might contradict our views and to change our minds as often and as readily as the evidence suggests. One "superforecaster" went so far as to write a software program that sorted his sources of news and opinion by ideology, topic and geographic origin, then told him what to read next in order to get the most diverse points of view.

The best forecasters are all curious, humble, self-critical, give weight to multiple perspectives and feel free to change their minds often. In other words, they are not (using Isaiah Berlin's iconic description, harkening back to Archilochus), "hedgehogs," who explain the world in terms of one big unified theory, but rather "foxes," who, Tetlock explains, "are skeptical of grand schemes" and "diffident about their own forecasting prowess."

But as Tim Richards has argued, we are both by design and by culture inclined to be anything but humble in our approach to investing. We invest with a certainty that we've picked winners and sell in the certainty that we can reinvest our capital to make more money elsewhere. We are usually wrong, often spectacularly wrong. These tendencies come from hardwired biases and also from emotional responses to our circumstances. But they also arise out of cultural requirements to show ourselves to be confident and decisive. Even though we should, we rarely reward those who show caution and humility in the face of uncertainty.

One forecast we should invoke, at least until the evidence demonstrates otherwise, is that the markets will generally trend upward. According to University of Oregon economist Tim Duy, "As long as people have babies, capital depreciates, technology evolves, and tastes and preferences change, there is a powerful underlying impetus for growth that is almost certain to reveal itself in any reasonably well-managed economy."

Because of the many problems we have with forecasting generally, our portfolios should also be diversified. A diverse portfolio — one that reaches across market sectors — greatly increases the odds that at least some of a portfolio's investments will be in the market's stronger sectors at any given time — regardless of what's hot and what's not and irrespective of the economic climate.

At the same time, a diverse portfolio will never be fully invested in the year's losers. For example, according to Morningstar Direct, about 25% of U.S. listed stocks lost at least 75% of their value in 2008 but only four of over 6,600 open-ended mutual funds lost more than 75% of their value that year. Thus a diversified approach provides much smoother returns over time (even if not as smooth as desired!). On the other hand, a well-diversified portfolio will always include some poor performers, and that's hard for us to abide.

Next, make sure your time horizon is long enough. If you don't have at least a five-year time frame before using the money, stocks are almost surely a bad idea. That's because the chances of negative returns over shorter time periods are too high. But over the longer term, our investment prospects are reasonably bright.

Finally, recognize that there are limits to how precise even good forecasting techniques can be. No forecast should be seen as more than the roughest of outlines. Your mileage can and will vary. Keep your attention focused squarely on specific needs, goals and what you can actually expect to control about your portfolio and its results.

Forecasting is a necessary element of financial planning and investment management. Yet we tend to be really bad at it. Thus handle your forecasts and your forecasting process with extreme care.