"Visit a trading floor today and you will hear the hum of servers"
In my experience, the machines doing the actual trades are as close as possible to the exchange, generally inside. What's in the trading floor are the client and ML programs that issue the strategies. That said, I've been away from the industry for a few years and it might have changed.
weak human + machine > machine > strong human
IE. Chess / Go / Dota 2
A better machine is just a tool.
One that someone who's smarter than you is going to use to win at the game of poker.
Hence Kasparov's law. You know, the chess guy.
One of the most surprising details in the whole book was the trading system devised by the biggest HFT-complainer, "Thor". And not even the system itself. Trading firms have to connect to practically all the exchanges, because they need the ability to send in orders directly to one of them. In trading, adverse selection is a thing - if an institutional trader sends in a big order, liquidity providers (HFT firms) can get hammered pretty badly. So when someone is buying or selling in one exchange, the HFT shops will adjust (read: pull) their quotes on other exchanges before the same executions would hit them there as well.
The basic assumption of a liquidity provider can be summed up as: if someone is executing trades, they must know something more; the prices currently on offer are clearly wrong. Pull quotes before things get expensive.
Thor wasn't exactly an optimisation engine, it was more a synchronisation engine. It kept track of transmission latencies across all the exchanges and could coordinate order creation times to such a degree that the orders that it wanted executed would land in all the exchanges at almost exactly the same time. This would allow it to execute its orders on all the exchanges at the prices available at that very time, without giving HFT firms the time to communicate across exchanges to pull their quotes. If I remember correctly, even the book used the term "slam" for the behaviour.
Thor used a strategy that any half-decent engineer should come up with in less than 5 minutes. But it was considered unfair by all the other market participants. The book didn't tell much more about Thor, other than that its use was discontinued shortly afterwards.
Rather amusing, nonetheless.
I suspect people can't tell the difference between the arbitrage that will happen no matter what (Chicago to New York, etc.) with the front running that came from seeing orders too large for one exchange and buying the rest from the other exchanges so they could sell it to the original buyer.
But Lewis realized that the idea of HFT actually helping the little guys and hurting the big guys wouldn't be as sensationalist and provocative a story, so he decided to twist the facts to suit his own narrative.
Trading is cheap now... because it's all been computerised. The exchanges are digital - as soon as you can do all the transaction processing by computers there is an entire army of (expensive) clerks that can be laid off who used to handle the paper work. HFTs occupy a niche where they can exploit latency between exchanges for profit, with occasional other activities that were considered to be far more questionable when it was people playing those games, not algorithms.
This is true but the (fairly recent) move to essentially free trading for retail investors exists because firms will pay money to brokerages like Robinhood just to get the opportunity to trade with a pool of traders that are unsophisticated.
So why would people pay for order flow? The people who do this are liquidity providers. What they want to do is sit there and buy a stock for $10 and sell it for $10.01 all day long. If they are buying and selling to you or me that is a great deal.
But what happens when BIG HEDGE FUND buys a shit ton of stock for $10. BIG HEDGE FUND probable knows the price is wrong at which the liquidity provider will sell a bunch of stock for 10.01 but then instead of being able to buy it for $10, all of a sudden has to buy it for higher. This is how liquidity providers can lose some of their profits.
So they pay brockerages like Robinhood that don't have big hedge funds as customers, not to fleece those customers but just to sell liquidity to them without worrying about getting picked off.
Quantitative investing as a whole is becoming more and more popular, while HFT isn’t.
Moreover, very few funds pursue completely automated strategies. Quantitative giants like DE Shaw, Bridgewater, and Jane Street still execute large amount of discretionary human trades. Of course these trades are informed by models to a large degree, but if that’s the definition of automation, the markets have been automated for over 50 years.
2) Even today, bonds are traded OTC and mostly by people. In the 1990's, 100% of bonds were traded by people. Bond market cap dwarfs equity market cap.
3) Exotic OTC options were a big thing in the 80s and 90s. While that's not true today, when they existed, all of those were traded by people as well.
4) Most hedge funds today manage their positions with excel and trade based on analyst recommendations. They might have some fancy factor ops engine or something, but it's basically an after thought.
5) Around the dotcom boom, Goldman's equity desk had over 500 traders.
Fundamentally we are talking about a lot of different things. Different asset classes, buy-side vs sell-side, NYSE systems vs pit traders, etc etc. We can slice and dice these numbers in many different ways: total number of discretionary firms vs quant, trade volume vs assets, buy-side vs sell-side, etc etc. There's many ways to look at it that result in different conclusions. All I'm saying is that computerized trading is probably not yet the dominant paradigm and it certainly wasn't the dominant paradigm in 2007, much less 1990.
If the rules/algorithms are set and monitored by humans, the humans are the ones ultimately responsible. Computers are just tools which provide greater utility and control for investors than ever before.
That comes with its benefits and also with its potential downsides, but does not lessen from the fact humans are ultimate responsible.
One of the points that the article makes is that this statement is changing, that computers are increasingly creating their own rules. The literal next sentence:
"New artificial-intelligence programs are also writing their own investing rules, in ways their human masters only partly understand."
You're right though, that the responsibility still falls on the humans. Anybody running algos that they don't understand should ensure that they are covered from a legal/ethical perspective.
Apparently, none of the Nobel prize winners thought to model out that particular adverse condition.
Bunch of smart people + one missed market condition = failure (eventually).
If there aren't people involved at every step of the way to inject humanity into decision making processes, it's much easier for companies to hide behind their "algorithms."
I believe the blossoming buzzword for it may be Mathwashing.
In my experience, the opposite happens.
These are all public. Beyond that I can't say :)
RenTech is also known for hiring/poaching almost everyone from a former IBM voice research team.
Leonard Baum of Baum-Welch/EM algorithm fame (for training HMMs) was one of the first recruits to rentech. They pretty much scooped up the entire IBM speech and language processing team (Mercer's 1980 paper is cited as eqn 1 in the GPT-2 paper). Robert Frey, formerly of rentech, confirmed their past use of NNs.
I think machines investing is less than 1%;
Of course this only US and only equities, but it's still well above 1%.
This is quite the sensationalist bullshit. What does this even mean? The rest of the article is paywalled, so I don't know for certain where the author is going with this, but what mayhem is there to speak of?
As noted by other commentors, retail investors are much better off these days because of tightened bid/ask spreads.
After working in HFT for nearly 10 years I've learned the firms that do the best do so by proving value to the market. Through competition with one another, retail investors are much better off by getting better prices from market makers who do a better job.
Also I think this article is overstating the prevalence of quant strategies and funds. At my old portfolio analytics firm, we had hundreds of hedge fund clients and not a single one was doing fully automated trades. Of course they had optimization engines and tech to squeeze a few bips here and there, but the fundamental ideas were still generated the old fashion way (through sleep deprived analysts).
You misspelled 'material non-public information'.