Social media and the stock market

Social media such as Twitter helped foment the Arab Spring and the London riots. But can they also help investors predict stock prices? asks Simon Wilson

When was this link first identified?

Can web-based data from search engines and social media be used to predict future behaviour? Interest in this idea took off after the publication of a paper in 2009 called Predicting the Present with Google Trends. In it, Google economist Hyunyoung Choi and her boss, the firm's chief economist Hal Varian, describe how peaks and troughs for particular products and services such as cars and holidays precede fluctuations in the sales of those products. In some sectors, the authors find an effect of only a few percentage points, but in others it's more substantial an 18% improvement in the predictions for the demand of motor vehicles and parts' and a 12% improvement for new housing starts'. Other academic researchers have found that mentions of political candidates on Twitter are good predictors of electoral outcomes. And firms such as Infegy have made social media-monitoring a buzz sector in branding and marketing.

Doesn't that sound a bit faddy?

The Bank of England doesn't think so. It revealed in June that it has been analysing data on Google searches for key economic terms as a way of tracking the economy in real time and the results are pretty extraordinary. Two bank analysts, Nick McLaren and Rachana Shanbhogue, have discovered, for example, that current searches for job seekers allowance' are closely correlated with official unemployment figures gathered by the Labour Force Survey a month later. The Bank also found that trends in searches for estate agents' may be a better guide to house-price changes than established surveys by the Royal Institute of Chartered Surveyors or the Home Builders Federation.

Subscribe to MoneyWeek

Subscribe to MoneyWeek today and get your first six magazine issues absolutely FREE

Get 6 issues free

Sign up to Money Morning

Don't miss the latest investment and personal finances news, market analysis, plus money-saving tips with our free twice-daily newsletter

Don't miss the latest investment and personal finances news, market analysis, plus money-saving tips with our free twice-daily newsletter

Sign up

How is this information useful?

It gives the bank's monetary policy committee an early insight into where the economy is heading. As the two Bank of England economists put it, the Bank's quarterly bulletin's "official economic statistics are generally published with a lag" of one month, three months, or even longer. Analysis of search trends, by contrast, is immediate. That's especially useful where data takes a long time to arrive or is of questionable quality. In Italy, two economists affiliated with the Bank of Italy, Francesco D'Amuri and Juri Marcucci, also found that analysis of Google data improves unemployment nowcasts' and forecasts.

What about Twitter?

An academic paper was published in February with the self-explanatory title Twitter Mood Predicts the Stock Market. Three computer scientists Johan Bollen, Huina Mao and Xiao-Jun Zeng (two of them at Indiana University and one at Manchester) set out to examine whether societies as a whole experience mood states that affect their collective decision-making and whether these correlate with, and can predict, economic indicators. They did this by analysing the text content of tweets using two mood-tracking tools, OpinionFinder (which measures simple positive versus negative mood) and Google Profile of Mood States (which measures moods in terms of six positive dimensions; namely: calm, alert, sure, vital, kind and happy).

What did they discover?

Unless you are a computer scientist with a particular expertise in statistical analysis, you may well find the guts of the paper hard going (anyone for "bivariate Granger causality analysis"?). However, it boils down to whether measured Twitter mood states predicted changes in the closing value of the Dow Jones a few days later. The answer is yes 87.6% of the time. In particular, rises and falls in instances of "calm" words correlate with rises and falls in the stockmarket between two and six days later. Critics might think this isn't good enough: after all, it is more than possible even if you trust the power of fuzzy neural networks to be right 87% of the time and still lose lots of money on the stockmarket. Derwent Capital Markets, however, disagrees.

Who are Derwent Capital Markets?

Lots of hedge funds are rumoured to be using social media analytics to guide their thinking. Derwent is the only one so far to base an entire fund on it the Twitter Fund. Derwent is a small London-based investment firm that has signed an exclusive deal with the authors of the Twitter paper and in May launched the first hedge fund to base its decisions principally on "sentiment derived from real-time social media data analysis".

Paul Hawtin, the founder and fund manager, says: "For years investors have widely accepted that financial markets are driven by fear and greed, but we've never before had the technology or data to be able to quantify human emotion. This is the fourth dimension". Wish him luck.

Is this the future of forecasting?

The most striking and convincing research in the emerging field of nowcasting', suggests Tim Harford in the FT, was published by Nikolaos Askitas and Klaus Zimmermann as an IZA working paper in February. Askitas and Zimmermann used data from a new system designed to collect road tolls: most heavy vehicles in Germany have GPS-based technology that tracks their movements and levies the right toll. "Unsurprisingly, their toll index' provides a very effective "nowcast" of German industrial production. Perhaps we can look forward to the day when real economic data is collected and disseminated almost as easily as the charts on business television."