Book Review: Is Forecasting A General Skillset?

Superforecasting: The Art and Science of Prediction - Philip Tetlock and Dan Gardner

Expert political forecasting hasn't been great these last few years. It isn't just the improbable rise of Trump, it's worldwide-elections and referendums (Brexits) of all kinds throughout the world lately have confounded pollsters and predictors almost uniformly.

Why? Maybe this book has a few answers. Superforecasting is essentially a follow on to Philip Tetlock's renowned book on political forecasting: Expert Political Judgement, which at the time of its release was groundbreaking in its thesis that-wait for it-the opinions of political pundits were no better than anyone else's. Tetlock showed political forecasters are habitually wrong, yes, but he also delineated how they keep their jobs by highlighting the times they're right, shunning the times they're wrong and being vague. Very few of them have verifiable "track records" and they seem to like it that way.

Replace "political" with "market" and it's a near perfect fit for how much of the investing industry operates, too.

Superforecasting asks a more ambitious question: Is there such a thing as general forecasting as a skillset? Most of Tetlock's answer is derived from his Good Judgement Project-a largely anonymous online tournament of average folks and professionals who forecast everything from markets to geopolitical outcomes. It's a hoot and completely fascinating, and I encourage everyone interested in investing to give it a try, if for no other reason than it will teach you just how difficult this business truly is.

From mounds of data compiled studying the real-time results of the Good Judgement Project, Tetlock assembles generic lessons of what made the most successful forecasters. And, much to my delight, many are lessons we've touted here at MarketMinder for a very long time. Scaling, use of probability, contextualizing appropriately, knowing you could be wrong, the humility of knowing how little can truly be known about the future, cognitive biases, emphasis on empiricism over intuition-these are the traits sound judgement is built on, and yes, it's agnostic to the field: These are true for politics as much as markets.

Within all this is a message most will miss: Tetlock found that simplicity is essential to forecasting. Consistently, the best forecasters don't use sophisticated statistical approaches, and anything beyond basic data crunching is usually a misapprehension of reality. Reality is vastly complex; but to churn its possibilities through a human mind requires the disciplined arrow of simplicity. Otherwise, we wander and lose the thread of reality.

And on that basis, shockingly (but as Ken Fisher has argued forever), average folks can be just as good as or better than professionals at forecasting. Most of the best forecasters at the Good Judgement Project were average folks with little or no special training. Why? Pedigree and technical schooling are worthless in a mind that isn't disciplined and in control of itself. Investing is temperament as much as insight. Always has been.

Good forecasters, in Tetlock's view, are foxes versus hedgehogs, going back to Isaiah Berlin's famous essay. (Foxes are generalists in their knowledge; hedgehogs are specialists.) Investing is a polyvalent, polymathic enterprise-you need to touch upon statistics, economics, history, psychology, political science, to name a few. Yet, what we train today are specialists in narrow fields who generally can't see the bigger picture. Forecasting things like politics and markets is about seeing very big pictures. It's another reason we stress global thinking above all.

This book is without question among the better ones out there on forecasting as a skill. But let's not go overboard. Often, the prose descends into what I refer to as a "feel smart" book. There are a lot of them out there, mostly found in airport bookstores. There is a veneer here of high intellectualism, but actually this book is written for the "read it on a flight so I can feel smart over a few hours" type. 50% of the book is stories and anecdotes. Which isn't necessarily a bad thing, but be warned.

Also, Tetlock offers no "how to." He highlights methods and traits of good forecasters, and then acknowledges it's often as much "art as science." He points to a kind of personal, experiential importance to forecasting: "Foresight is not a gift but rather a product of a particular way of thinking. Superforecasters are open-minded, careful, curious and self-critical. They make an initial prediction and then meticulously adjust this prediction based on each new piece of related information." Along with this conclusion being basic, and sound, Bayesian philosophy, I'd imagine it also greatly angers today's high priests of number crunching. (Which makes me happy.)

Here's the rub: From the generic skills of what good forecasters have in common, at some point you must move into the particulars. And so, if you're going to try market forecasting you need to be acutely aware of how markets work-that is, the basic pricing mechanism. It's far different than any other kind of forecasting. A price is a piece of information about a security indicating prevailing opinion across all market participants. To discount, or "price in," therefore means surprise power only lies in what isn't widely recognized or already reflected in market knowledge. This means forecasting stock markets is frequently counterintuitive, different than the consensus-to know where markets go next you have to think in terms of what they haven't yet priced in. Without that basic insight the generic lessons of this book won't get you far.

Lastly, Tetlock encourages forecasters to be as precise as possible in their forecasts-to actually say something like "there is a 73% chance" of XYZ happening versus saying it's just likely. He believes this promotes development and introspection from experience and learning from mistakes. Which is fine, but in the world of investing it's dangerous. It doesn't matter whether the market is up 12% or 16% in any given year; what matters is that it's up, and in sufficient magnitude to make certain portfolio themes function the way they ought to. To spend time trying to figure out the difference between a few percentage points of return is to miss the forest for the trees and expend tremendous resources on effectively nothing. Markets are too noisy and full of chaos to try and pin down with such exactitude. As with long-term financial goals, the idea in market forecasting is to find the highest probability outcomes and point yourself in the right direction for as long as possible and in highly disciplined fashion. Exactitude blurs and fuzzies that basic aim.

If you're interested in forecasting, this one's well worth the time and effort. Now as for that election, I can make this forecast with the highest certainty: We'll have a new president come January 20.