Big data paper shows stock market trades behave like light bulbs

Once you remove the noise from a huge sample of market data its patterns become surprisingly predictable

stock exchange

In the stock market, electronic trading (AKA “etrading”) originally started so people could buy and sell stocks and other financial instruments more easily. No more hanging out on the floor of a stock exchange or calling your orders into your broker; you could do it all from your desktop. This was good because it made markets more accessible and reduced costs.

Then, in the 1980’s, because the electronic trading platforms had application programming interfaces to allow new client-side interfaces to be developed, the inevitable happened, the next generation of electronic trading appeared. Algorithmic trading (AKA “algo trading” or “black box trading”) removed humans from the equation and exploded as the latest, greatest stock market money-making strategy. 

Computers take over

Algorithmic trading involves computers running complex statistical and predictive algorithms interfacing with exchanges to buy and sell at lightning speeds, something that has become termed “high frequency trading”. As of 2012, it was estimated that roughly 85% of all market trades were algorithmic. According to Wikipedia:

High-frequency trading has taken place at least since 1999, after the U.S. Securities and Exchange Commission authorized electronic exchanges in 1998. At the turn of the 21st century, HFT trades had an execution time of several seconds, whereas by 2010 this had decreased to milli- and even microseconds. Until recently, high-frequency trading was a little-known topic outside the financial sector, with an article published by the New York Times in July 2009 being one of the first to bring the subject to the public's attention. On September 2, 2013, Italy became the world's first country to introduce a tax specifically targeted at HFT, charging a levy of 0.02% on equity transactions lasting less than 0.5 seconds.

Enter the flash crash ...

As with any sufficiently advanced technology (whether it appears to be magic or not), there have been unforeseen consequences, once again confirming the old adage that “to err is human but to really **** ** requires a computer.” 

On May 6, 2010, there was a “flash crash” and the Dow Jones Industrial Average fell by almost 1000 points then recovered within the space of 15 minutes. The result was chaos and investigations but the constraints on algorithmic and high frequency trading weren’t enough:

In October 2013 a flash crash occurred on the Singapore Exchange which wiped out $6.9 billion in capitalization and saw some stocks lose up to 87 percent of their value … resulted in new regulations … to curb excessive speculation and potential share price manipulation [and]  Two short-lived (less than a second) movements (more than 1%) in several (40 and 88) stock prices followed by recovery were reported for November 25, 2014.

And the long term consequences of flash crashes are significant; for example, according to a 2015 article on TheStreet:

Procter & Gamble's (PG) stock chart … reveals the low price that Procter & Gamble shares reached during the crash, in which the Dow Jones Industrial Average plunged 998.5 points in a matter of minutes. …   Many of the components of the blue-chip index had their lowest trades canceled as being unrealistic, but Procter & Gamble's remains. The chart shows that P&G traded as low as $39.37 on May 6, 2010. … That low price is important, because it can skew how technical analysts read Procter & Gamble's chart.

In other words, had the 2010 flash crash not happened, P&G’s stock would be valued better than it is today.

Analyzing algo trading

The reality is that algorithmic trading isn’t going to stop and there’s a limit to what technical curbs and regulations can be defined and enforced. This leaves us with the lingering problem that electronic trading can be used to manipulate the market and the astounding volume of trades (on July 22, 2016, on the New York Stock Exchange alone, 750,899,588 trades were made) makes detecting anomalous and possibly fraudulent trading activities extremely difficult. At least, until now …

A recent paper, Universal behaviour in the stock market: Time dynamics of the electronic orderbook, by Ayşe Kızılersüa, Markus Kreerb, Anthony W. Thomas, Michael Feindtd, in the journal Physics Letters A,  revealed:

… the timing of electronic orders on the stock market can be mathematically described in the same way as the lifetime of a light bulb. The surprising finding is a “crucial first step” towards predicting dramatic movements on stock exchanges that could lead to stock market crashes.

The data was from four months of buy and sell “limit orders” for seven stocks on the London Stock Exchange (definitely a Big Data problem). Professor Anthony Thomas, Australian Laureate Fellow and Elder Professor of Physics at the University of Adelaide, and one of the paper’s authors, explains on The University of Adelaide’s news page:

We found that when we looked at orders that came in extremely close together, less than 10 milliseconds apart, there were a huge number of orders placed and withdrawn that don’t satisfy any rational formula that we could see at all …  It appears that in these cases, what’s going on is some attempted market manipulation through fake orders to try and suggest that the market is moving when it’s not. However, when we excluded all the orders of less than 10 millisecond intervals, we found the market actually shows amazingly rational behaviour. In fact the pattern of placement and removal of orders then follows a well-known probability distribution, the Weibull distribution.  And even more surprisingly the shape of the distribution is the same for all the stocks we studied – a shape that corresponds to ‘maximum entropy’ or, in other words, maximum disorder.

I won’t attempt to explain what a Weibull distribution is (mostly because I only barely understand it) but what matters is that it can model many different data sets. In the case of the paper’s stock market data, the authors point out that the model looks similar to the distribution of light bulb failures over time. This is important because once you have a model that characterizes your data, in this case, from the stock market, it becomes much easier to predict and detect anomalies such as market crashes and stock manipulation.

The researchers plan to next study price movements. 

Comments? Thoughts? Email me your analysis and predictions or comment below then follow me on Twitter and Facebook.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10