Book Review: Signals, Noises, Foxes, Hedgehogs, and Bayes

The Signal and the Noise – Nate Silver

I’m completely fascinated by forecasting—the very idea of it. Weather, politics, stocks, you name it. I’ve studied statistics and probability theory, chaos and complexity theory, economics, finance, physics and meteorology—all from curiosity about forecasting. Even blind Greek soothsayers (augury is among the oldest professions).

This book reviewer has over the years highlighted a few themes germane to the disposition of good market forecasters: Bayes’ theory, being a fox instead of a hedgehog, and recognizing the economy as a complex, adaptive system. (*See below for brief explanations of these concepts.)

Renowned political forecaster Nate Silver, in his first book, has unified these concepts into the best work on forecasting as a broad topic I’ve ever read.

The book’s main purpose is to emphasize what useful forecasting information really is. Silver wants us to understand statistics alone are not good enough for consistent, accurate prediction: Subjectivity and qualitative reasoning are just as vital. In fact, he goes so far as to call the “narrative” or story of the situation as equal to slavishly examining numbers. For him, this is the key to separating the signal from the noise. This is the heart of Bayes’ theory of probability, and also reminds me of younger days when Ken Fisher constantly admonished the Fisher research staff, “No correlation without causation.” That is, even if you discover a statistical relationship, you still must understand and explain it—otherwise, it’s not actionable as an investing idea.

From this stems Silver’s most provocative insight: He believes there is no such thing as a purely objective prediction—subjectivity will always play a role. In today’s world of “big data,” “quants” and statisticians as soothsayers, this is crucial. Simply, experience, reason and general quality matter in the forecasting business.

To wit, Silver details many common faults of forecasting. (MarketMinder has described most common psychological biases over the years, so no need for that here.) But a common pitfall that popped out was the idea of “over-fitting” a prediction. What this means is being too tight and too precise to be useful. Over-fitting looks superficially more impressive but is in reality less effective because to be overly precise requires listening to some of the “noise” and not purely the “signal” of information. Thus, Silver sees most economic indicators and data like GDP as "blunt" instruments at best. This is indubitably true: Far too many forecasters get caught up in making a precise forecast for the stock market when really the thing that matters most is what direction it’s headed.

In the world of constantly increasing content, Silver makes the case that yes, information is increasing, but “useful” information remains relatively constant. That is, the “noise” is increasing and there is more of it to sift through. This might seem trite, but still today the investing public pays far too much heed to the sensational story of the moment; and the financial press loves to feed that fire.

So, Silver advises us to ask for the motives of those who are giving predictions: Do they really have an incentive to be right or do they have an incentive to sound smart? Or to appeal to a certain type of audience they know will want to agree with them? Here, Silver relies heavily on Phil Tetlock’s seminal work on political forecasting, also a recommended read.

Economic forecasting suffers from a few primary faults, in Silver’s view. First, it’s difficult to untangle cause-and-effect even in retrospect unless you actually go back and study the history of the situation. Second, because the economy is dynamic and always evolving and changing, it becomes difficult to compare the past to the future. This makes understanding current context all the more important. Lastly, economic data are not very good. These lead to the aforementioned problem of correlation without causation. There are basically no independent variables in economics, making much statistical forecasting suspect or even irrelevant. Recessions, for instance, are such rare events that most empirical analyses of them are vastly imperfect and probably wrong (even today, economists fight over the “true” cause of the Great Depression).

Silver believes the best forecasters are wrong fairly frequently but value remains in the doing of the forecasting for both society and/or the common investor—good forecasters are right far more often than the layperson or non-expert. Learning from past wrongness has always been a hallmark of good forecasters: Silver habitually studies his past predictions and asks himself what he could have done better or differently. All forecasters should. Interestingly, one way he decides if he made a good forecast even if he was wrong is if he’s “at peace” with his analysis. That is, maybe he was wrong, but did he make the best possible forecast with the information available to him at the time? It's the same for investing: The best forecasters will be wrong fairly often but if the process is rigorous and the analysis is accurate, over the long run they will be right far more often than not. Too many folks become overconfident if they happen to be right about a prediction but for the wrong reasons. This is epidemic in perma-bears out there today.

Silver believes in the wisdom of crowds: They aren’t always right but they’re frequently more right than individuals. He hails the overall effectiveness of prediction markets companies like the now defunct Intrade website, but notes they must grow hugely in number of participants and volume traded before they’re truly efficient. (What a shame it is Intrade no longer exists.)

Silver is something of a polymath, and his book often shows it. A favorite passage includes a fascinating comparison of philosopher David Hume’s work with Adam Smith and even Bayes. Silver asserts that Bayes’ theorem and Smith's economic ideas (specifically the “invisible hand”) are really part of the same tradition and that both rely on the wisdom of crowds to solve many problems. This is a penetrating and unifying insight on an old notion.

*Three of my favorite forecasting heuristics:

1. The importance of Bayes’ Theory: mere statistics is not enough when creating probabilities; context and fundamentals must also be subjectively considered.
2. Being a fox and not a hedgehog: knowing a little about a lot of things in life versus knowing a lot about one narrow thing. And that understanding an economy or market is often not quantitative but can be more like a qualitative narrative.
3. The recognition that the world and its economy is a complex system: the world is not linear, it’s not easily reduced to mechanistic and deterministic equations, but it does have tacit cycles and frequently recurring features.