Investment Research Gone Wrong

To get an investment edge, you need either information others don't see, or a better interpretation of widely available information. This is the purpose of investment research, whether in finance journals, mainstream news publications or Wall Street-focused blogs and forums. Sadly, investors turning to these sources for actionable research often find shoddy data and shaky conclusions instead. To help you separate out the noise, here are three forms of dubious investment research to watch out for.

The Tortured Data

Ever see an investment pitch declaring that the data have spoken, and such-and-such an investment strategy is a proven winner? Or why this one metric can predict market movements? The charts and statistics accompanying such claims may seem objective and ironclad, but they often suffer from what statisticians call "p-hacking"^[i]-cherry-picking, slicing and dicing data until it yields a "statistically significant" result.^[ii] Crunch enough numbers, chart enough trends, tweak enough equations, and you're practically guaranteed to find some apparently rock-solid connection. But correlation without causation is a big booboo! There must be a sound, fundamental "why" behind the link, otherwise, there is no good reason the connection won't fall apart.

Using historical market data to prove a strategy's effectiveness-known as "backtesting"-is another way to torture numbers. Backtesting is often used to support "smart beta" strategies, which are supposedly "passive" products based on alternative indexes, with firms selected and weighted according to certain valuations. While the strategies are new, many boast "historical" data showing how the strategy would have done in the past. Trouble is, this is all hypothetical, and it's fairly easy to tweak certain variables (like timing and frequency of index rebalancing) in order to get a better outcome. That doesn't mean much for the future. Some backtests suffer from survivorship bias, where firms that go bankrupt are excluded from the final performance calculation since they weren't in the index at year-end-but a real-world investor still would have felt the loss. Always remember-you're buying the uncertain future, not the known past.

The Plausible-Sounding Theory

Some popular investment tips are based on connections that make intuitive sense but don't withstand scrutiny.

One recent and illustrative example is the "HQ Indicator." This holds that companies that shell out for gleaming new corporate headquarters^[iii] are great "Sell" candidates, because such profligacy is a sure sign of malinvestment and profit-killing hubris. Sounds plausible! But it's based on anecdotes and intuition-a few poorly defined examples over a narrow timeframe, held together by a general feeling that this should make sense. The same could be said of recommendations to buy companies where insiders are buying, exciting new products are hitting stores, or that announce a higher dividend or stock buybacks. The stories may be compelling, but they don't hold up-either because the indicators in question don't actually reflect a company's health, or because forward-looking markets take positive (and negative) news into account immediately, negating the benefits of acting on it.

To identify this form of subpar research, keep an eye out for the following: Narrow or unspecified test periods purporting to show an indicator's reliability; small sample sizes-often just a couple anecdotes; appeals to emotion or political leanings, which make us turn off our reasoning; and a reliance on narrative over facts.

The Anonymous Tip

Have you ever seen blog posts all about one hot penny stock you've never heard of, written by someone with a name like "Bodacious Stock Bro" or "Stock Surfin' Sally"? (Yes, we know those are terrible, sorry.) These may be the work of paid stock promoters, who are, well, paid by the company to boost interest in their shares (which are usually thinly traded stocks trading at only a couple dollars or a few cents). It's legal, as long as they disclose the arrangement, but that doesn't always happen. In the pre-internet days, paid stock promoters would spam brokers and financial advisers via fax with "research" pumping up a particular stock. Now, they can go right to the public, using blogs that will publish virtually anyone, and posting under flashy pseudonyms. It seems some also didn't disclose their role. The SEC recently fined some companies and writers (including the delightfully named Trading Maven and Wonderful Wizard) for failing to disclose payments for positive online coverage of various companies.

Don't get hoodwinked by a persuasive pitch, especially from an anonymous source. Maybe it is an expert! Or maybe it's a bored 12-year old with zero knowledge. How can you know? And if you can't know, how can you evaluate the opinion and quality of analysis? In our experience, anyone with actual expertise and solid credentials is happy to put their name or firm's name on it, while those hiding their names could be hiding something else, too.

Ultimately, no form of research consistently yields an investment edge. But smart investing is often about avoiding mistakes-not falling for chance correlations, weak narratives and conflicted recommendations. Thus, you should demand more from others' research-and your own. Question assumptions. Seek out missing premises. Probe suspicious data. Ask yourself: If I had to disprove this, how would I? This sort of investigation, not dodgy research, helps investors most.