Hacker News new | past | comments | ask | show | jobs | submit login
A Gazillion-Dollar Standoff over Two High-Frequency Trading Towers (bloombergquint.com)
173 points by jakub_g 15 days ago | hide | past | web | favorite | 193 comments



I was working in HFT myself, though in Europe, and I asked around about this. Specifically WTF do they do with this line from Chicago to New York that's so insanely important?

The US market has this interesting phenomenon that the futures trade in Chicago, but the single stocks trade in New York. For those who are not familiar, this means you can make a bet on the S&P 500 index as a whole in Chicago by buying or selling futures. But if you wanted to replicate the basket, you would go to New York.

Naturally there's a relationship between the two. If the futures are going down, the prices of all the stocks needs to be adjusted. And if you look at stock prices, it turns out quite a lot of any stock price is simply exposure to the market at large. So you really want to know as soon as possible if the market is moving.

Now you might have thought that the index is the index of shares, which is in NYC, so why does it matter what Chicago thinks? Well actually the tail wags the dog in this case. The futures are how people express a view on which way the index is going, and you want to know that view immediately.

How does it actually work? Variants on "if the market goes down, pull all your bid orders". Basically if things are fine you are happy to look at queue position and imbalance as an indicator of whether you want to leave orders to capture spread with. But the moment there's an indication from Chicago that the market is dipping, you forget that and pull your orders. Or if you're really fast dump a sell order on the guy behind you in the queue.

There's a huge amount of stock trading in NY each day, so it's worth your while to have this line that you use to send the warnings down.


It's basically arbitrage. Information and time difference arbitrage. Different exchanges on the East Coast, because of their different locations receive trade info at different times. This is what they are trying to capitalize on. And also, the obvious race to first receiver of information from Chicago.


Do the companies that do this make any claims about how they benefit society? To me, stock traders in general seem a departure from the idea that money is a result of providing value to society, but arbitrageurs seem even further detached. I'd be interested to hear if there are arguments I haven't thought of.


High frequency traders create liquidity. Arbitrage in general helps ensure that prices are uniform across geographies and across linked securities (e.g. options are priced appropriately vs. the underlying asset). You can argue that there is no incremental value in going to shorter and shorter timescales, but fundamentally this mechanism is how information moves around the economy.


There used to be less sensitivity to the receive-side time by exchanges, but in many cases there had been an emphasis on fairness and high res timestamps (they sell colocation services). Now, many times the first through the post wins. This creates a constant arms race and narrows the field to a handful of competitors. Many companies that survived in the fat tail can no longer do so.


A guy is willing to buy some shares at 101 on exchange A. Another guy is willing to sell the same shares at exchange B for 99. Neither of them has the budget to connect to both exchanges.

HFT has connections to both and can see the cross. Everyone is better off.

This might be different in the US due to NMS, but HFT strategies are all pretty similar: they can provide liquidity better if they are fast, and other people can take advantage of that.


I'd be curious to hear some statistics on how much money a ~10ms difference in time of information received can actually make.


I don’t have any statistics but I have an anecdote.

I built a trading platform ~6 years ago that had a 100 microsecond tick to trade (a term that means from the time a market update hits your nic to the time an order leaves your nic).

We knew then that we were too slow for large swaths of the HFT space & wouldn’t even attempt those trades.

I haven’t kept up with the industry but I’d guess tick to trade times at the speed edge are in the low hundreds of nanos now. In an environment like that a 1ms time improvement is out of the capabilities of your software/hardware engineers and can only happen at the network.


If we’re really talking “at the speed edge”, low hundreds of nanos would be already considered too slow these days :)


Hold my beer ...

https://www.youtube.com/watch?v=L5cZaIZ5bWc

And that was more than 6 years ago, one ticker symbol only. You can imagine now.


Holy shit.

That gave me a much better sense of scale than hearing "a trade takes 200ns" or whatever. The engineering that goes into making this possible must be just insane.


If you’re an options market maker and you are 10ms slower, you legitimately will be obsolete in the market. You will be arbitraged against when stocks move and you are slow to update quotes. This happens all the time, and it makes it so that you have to quote so wide that you aren’t competitive. Milliseconds are the difference from being competitive and going broke in the industry.


10ms is way too long. Think about 3D graphics drawn at 60fps. That's 16ms you get for each frame. But 3D graphics calculates a LOT of stuff for your whole game in that time.

In trading you're not calculating nearly as much stuff.

In trading it's probably very sharp, the line between making and losing money. You want to be under 100us for sure. That's micros. Depends on what you're doing of course but 10ms is OOM to slow.


Following up on this... The blog post Islands at the Speed of Light ( http://www.bldgblog.com/2011/03/islands-at-the-speed-of-ligh... ) with a "global map of 'optimal intermediate locations between trading centers,' based on the earth’s geometry and the speed of light"

There's a similar article at When The Speed Of Light Is Too Slow: Trading at the Edge ( http://www.kurzweilai.net/when-the-speed-of-light-is-too-slo... )

The paper that both of these articles are based on: Relativistic statistical arbitrage https://www.alexwg.org/publications/PhysRevE_82-056104.pdf which opens with:

> Recent advances in high-frequency financial trading have brought typical trading latencies below 500 􏰒µs􏰃, at which point light propagation delays due to geographically separated information sources become relevant for trading strategies and coordination 􏰀e.g., it takes 67 ms, over 100 times longer, for light to travel between antipodal points along the Earth’s surface􏰁. Moreover, as trading times continue to decrease in coming years 􏰀e.g., latencies in the microseconds are already being targeted by traders, this feature will become even more pronounced.


I've seen this paper before, or something like it. It's incredible to me that anyone would publish this. It's like reading a paper on astrology or dowsing, written in the style of a physics paper.

You don't get any benefit from putting your trading system half way between the two exchanges, you colocate it with one or the other (or both). Likewise if there are three exchanges to trade on, you don't build a datacentre at the circumcentre of the triangle or whatever voodoo geometry nonsense. You colocate at each of the three and buy/rent/build a line for each of the three point-to-point connections. Source: worked in HFT for more than 15 minutes.


I also know one of the authors quite well, long term friend. He is nothing short of brilliant but also has a penchant for enjoying being a "subtle troll". While he does have a serious point/observation/discovery in the paper, I assure you, he's well aware of the "funny" or "goofy" aspect of the impracticability and laughing about it has been a source of great late night entertainment between us.


I know the author well and he would likely conced to you some of the practical points you’re making about collocation and market microstructure. However he wrote the paper to illustrate how global compute networks could someday collaborate using the techniques described to facilitate AI. Hft is a early form of AI so if that’s an area of interest for you maybe take another look at that paper.


I've heard the (facetious) comment, before, that if we ever do discover a faster-than-light theory it'll be because of HFT, not academia.


There’s also more straightforward reasons to want a microwave connection from NYC to Chicago. Say you want to trade off news signals: you want to relay the headlines (from Bloomberg, DJ, etc.) to your colo sites as quickly as possible.


Can anyone in the industry comment on whether this kind of strategy really exists? Surely any automated, news-based strategy's bottleneck is not due to sub-microsecond latencies stemming from microwave tower placement. I've always assumed that the strategies that require microwave networks are using market data and only market data as a primary input.


I can confirm first-hand that this is a real thing, and that if there’s significant movement immediately following a headline, you likely won’t end up with more than table crumbs if you’re not one of the first two or three players to respond.


Interesting. I’m curious, can you give an example of what kind of news? Some earnings reports or acquisition or something?

And are the reactions automated based on some NLP? Milliseconds wouldn’t matter if a human has to still parse it, correct?


Yes, NLP, and sometimes structured data released by special services that receive the news alerts early, process them into a structured form, and then make them available at the same moment that the normal services do.


Yep, NLP. As a grad student in comp linguistics, I was surprised to find out that financial news was an enourmous application space for NLP, for exactly this sort of reason. Including the automation of content creation. Much financial news isn't written by a human, and hasn't been for a long while. It's too slow. Faster content creation equals faster actionable data equals huge advantage.


NLP is a possibility, but bear in mind that HFT is all about optimization. The confidence levels desired can vary significantly based on what you intend to do with the information.

Acting on M&A news by taking a more aggressive position generally requires more confidence than, say, backing off based on ‘bad’ news. In the first case, you want to be pretty sure of the headline before taking action. In the latter, the mere presence of words like ‘fraud’ and ‘investigation’ might be enough to play it safe and say, “we’re going to back off trading this underlying until a human has a chance to look at this.”


or you misread the headline, price reverts and you end up with losing position. i am genuinely curious, how feasible this headline trading is.


I used to be in this business. It’s a good practice to think of everything in terms of probabilities alongside capacity/liquidity. If an all cash merger hits the news, there’s a race lift prevailing offers on the one stock as there is a much better expectation of value due to the news.

Now let’s say there’s negative news on the Chinese economy. Someone could be running an algorithm that sells down companies with Chinese exposure. But, inherently, there likely is a much lower probability that this news is uniform or material across companies with China exposure. A strategy like this likely would get put on small positions and get in and out quickly as it could get run over depending on how humans process the information and make bigger discretionary bets.

There definitely is a risk too all of this. Roughly 10 years ago, somehow an old united Airlines bankruptcy article hit the wire and the stock sold off hard. Eventually, United released a statement saying the article was innacurate. This isn’t typically the type of strategy you would make large, single bets with, or hold the positions over a long period.


There’s two sides to it: you might decide to take a more aggressive position, but you might also use the headlines as a signal to back off (and stop or scale back trading of securities for the affected underlying, at least temporarily).

One tends to carry more serious consequences when you’re wrong. Accordingly, the ‘take’ path generally has more checks, and more opportunities to abort.


>> or you misread the headline, price reverts and you end up with losing position.

Yes, you might misread here or there. But this is a game of statistics and the real win/loss is whether you read correctly more often this read incorrectly. You want your net wins to be positive.


Quite a few players are trading on economic releases (eg non-farm payrolls, ADP, ISM, and so on) in this fashion. It’s often much easier to feed to algo because the release is a number (or set of numbers) vs some free-form-text thing. Competition is at a sub-micro latencies today.


Yes, this exists.


Thanks for this explanation. I previously thought the distance fight was over being closest to the NASDAQ co-location building in New York, not connecting Chicago and New York.

I am still not clear why microwave is faster than fiber though. Is it simply that the fiber lines are multi-tenant, so there is not a direct hop?


Light in glass is slower than light (or radio waves) in air. Also where they're unable to put point-to-point microwave towers, like the Atlantic ocean, they use HF/shortwave (as in 3-30MHz stuff) radio, because frequencies in that range bounce off the ionosphere. And apparently, the added lag from the increased path length is less than the added lag from glass's optical index, so HF wins out. HF is, however, very noisy and the bandwidth (amount of bulk bits per second you can send) is not too high (even though the end-to-end latency for a single given bit can be very low).

It's not pure HF -- there's a fibre link alongside the HF link where stuff like protocol negotiations, encryption keys, channel sounding, feedback on data received and RF conditions, and agreement on what kind of modulation/coding schemes to use for upcoming data (and what transmitted bits are to mean in terms of higher-layer protocol (something about price informations? I don't know much about HFT)).

Likewise, the availability of GPS makes time/frequency synchronisation easier (and combined with sounding signals and feedback over fibre), which very likely allows for eliminating latency-inducing complexity in the receiver.

It's a far cry from the traditional data-over-HF scenario, which is assumed only to be relevant when other (higher bandwidth) links such as fibre or satellite or point-to-point microwave are unavailable, so everything must be carried out over the RF channel (look at ALE).



Also consider the path: you can pretty much do a great-circle with radio.

A cable is not going to be direct. If it's above ground, it needs some slack on the line to account for pole movement (you don't want your fibre holding two poles together...).

You'll need more slack for your splices to be done at surface level during construction. Splice trays every x meters will add more distance.

Sometimes the poles are overloaded and you need to run down and bury it for a while.

Your tunnels are never really straight. You need to follow existing easements, or get your own. When you do bury, you need to go around various physical obstructions, like rivers, buildings, cities, parks.

And if you want redundancy against back-hoe fade, your latency in those situations will shut you down, perhaps erasing your profits when you're down and your competitors are up.

The cost of running your own line point-to-point would be be monstrous, though I guess you could build in some latency and sell access to non-HFTers.


the index of refraction of a material is essentially a ratio to the speed of light in the material, where free space is 1. Glass, being around 1.5, has a lower speed of light.


So that's why my diamond optic cables never achieved product-market fit! http://hyperphysics.phy-astr.gsu.edu/hbase/Tables/indrf.html


That, and how the fick do you machine diamond... regular glass fiber optics needs to be cut to a smooth surface and fused together. The devices to do this use diamond-coated blades.

Oh, and diamond has a large birefringence due to its crystal structure, which would cause temporal smear in signals.


Fiber is about 0.6c and the cables are much longer than direct line of sight.


Microwave signals travel from the transmitter to the receiver in a straight line. Whereas light bounces off the walls of the fibre optics cable, which increases the distance. Furthermore, the material that fibre optics are made out of attenuates the signal much stronger than air, so you have to place more repeaters and electronics in between your endpoints.


> light bounces off the walls of the fibre optics cable

This is true of multimode fibre, but not single mode. Long haul communications cables are single mode.

https://en.wikipedia.org/wiki/Single-mode_optical_fiber


Even for multimode fiber it's a huge simplification of the actual physics. The actual cause of the speed difference, for both single and multimode fibers, is the index of refraction of glass.


Look up refractory index. That’s all you need to know. The above is meaningless in this context.


Light signals in an optic fiber bounce off the walls (repeatedly, frequently) of the cable they're in -- the actual physical path the light is taking in this case is approx. 1.5x the path of the cable itself.


That's not true. The typical total reflection angle in single mode fiber optic cable is less than 4 degrees. The refractice index of glass, however, is about 1.45. So almost all the slowdown is due to the glass directly slowing down the signal.


Though you can still get a group delay (slower than free space) in vacuum filled RF waveguide. When you break down the single mode, it appears to be zig-zagging through the waveguide (bouncing off the walls), and slows down as cutoff is approached. Like in fiber, the modal dispersion is separate from the material dispersion.

AT&T was going to build out a cross country, single mode, circular waveguide network in the 60’s, but was obsoleted due to fiber. Maybe that’s what the HFT ought to due, or maybe air-line coax.


I think the distance is less of an issue than the temporal smear that the bouncing causes. A clean square pulse turns into a blob.


Temporal smear is caused by the varying refeactive index of glass with wavelength and the fact that a short time pulse must have finite (large) spectral width (by the uncertainty principle), not by the bouncing.


So, the different path lengths implied by internal reflections don't contribute anything to the smear?

By the way, the uncertainty principle doesn't say we can't have a narrow pulse. Frequency versus time can be used as an analogy to help explain the uncertainty principle. If we know a precise frequency, the signal is poorly localized in the time domain. If a signal is narrowly localized in the time domain, it is poorly localized in the frequency domain. The uncertainty principle is something else, though: it revolves around not being able to make certain related measurements simultaneously, at the quantum level. We can be quite certain about the frequency content of a transient pulse and its exact temporal shape, if it's not something from the quantum domain.


Is this why people move to Pittsburgh and Cleveland?


The L1 repeaters along the path add very little latency and require no real presence.


Can someone explain to me the purpose of HFT? Not the economic incentive, I get that, but what does it accomplish? Is it a net benefit to the system? If we arbitrarily limited transactions to human speed would there be some kind of systemwide downside? Are there any good, human readable papers on the benefits of HFT?


The only kind of HFT I’m directly familiar with is for automated market making, so I’ll speak to that.

Market making in itself provides a valuable service by keeping spreads razor thin and ensuring there’s enough liquidity to satisfy almost any trade at close to the current spot price. HFT is mostly just a way for market makers to gain an edge over each other. In general, it neither helps nor harms ‘regular’ customers. It would hardly be noteworthy if not for the insane lengths that HFT groups go to in order to gain an edge. And, of course, the possibility that a HF system with inadequate controls goes on a trading rampage that significantly moves the markets (see Knight Capital).


> market makers

How many market makers are there?


A lot less than their used to be. The margins have been driven out of that business. Like most industries that’s led to consolidation.

I’d guess Citadel/Virtuu/Jump capture the vast majority of that business.


Also Jane Street and Sasquehanna in the EFT space. Both of those firms are making great money nowadays.


Yeah the more I think about it there probably are lots of market makers still because of market specialization. But it’s not like before when a single person could be a market maker or even later when small prop shops could do it.


Anywhere I could read about the historical context/progression you reference?


Wolverine is in there somewhere as well


Typo corrections: ETF and Susquehanna.


You can get a rough idea by looking at the number of firms in these links. Overtime, it has consolidated.

https://www.canarywharfian.co.uk/threads/list-of-proprietary...

Then also the associate firms on this list:

http://fia.org/members


> > market makers

> How many market makers are there?

However many the market will support.

How many shoe shops are there?


The HFT systems are replacing the people who did this manually. eg > At its height back in 2000, the U.S. cash equities trading desk at Goldman Sachs’s New York headquarters employed 600 traders, buying and selling stock on the orders of the investment bank’s large clients. Today there are just two equity traders left.

https://www.technologyreview.com/s/603431/as-goldman-embrace...

It makes the markets more efficient which is good for everyone.


Interesting. I did not know this. Thank you!

Do you think there was any benefit from having humans do this work? Was there any subjective analysis used, or were these 600 people just human calculators?


There was objectively not a benefit. In addition to the fact that trades took longer, in material amounts of time, they also cost orders of magnitude more.


I don’t think that is true. bid and asks aren’t a true/total reflection of the msrket. There’s not going to be many people with a bid of $5 for Apple, even though any rational being would buy Apple for $5. Human traders can access this hidden liquidity by picking up a phone or messaging someone on the terminal.


I'm sorry, I'm having trouble following what you're trying to say here about "bids of $5 for Apple". The bid-ask spread is the difference between the lowest available sell (asking) and the highest available buy (bidding), not the lowest buy and the highest sell.


I'm not talking about the bid-ask spread. A bid is just someone willing to buy something for a specific price. An ask is someone willing to sell something at a specific price. The bid-ask spread is the difference between the lowest ask and the highest bid.

The point I am making is that the order books of markets don't accurately reflect the real underlying demand for the securities.

Let's take a hedge fund, who would buy apple at $5. It's incredibly unlikely that they will actually place a bid on the order book at $5 for apple. Instead, they will monitor the price of apple and buy it if they see a good price.

The value that human traders can provide is that they can tap into this hidden sources of liquidity that isn't reflected on the order book of the exchange. They can call up some manager and say "hey I can sell you something at a good price, you interested?"

Without human traders, there is no way to tap into this liquidity. This occasionally causes problems in thinly traded markets or during after-hours trading.


Superficially they didn't add much. On a deeper level nobody really knows. The future could be all Index funds and bots trading historical patterns and could result in poor investment choices which screw up the real economy. The dotcom boom was a great example that this happens already, nobody is sure if this new world is better or worse.


Zero benefit. They were not like human calculators, but more like middlemen, mainly because access to the markets was difficult.

Now as everything has become more efficient, humans are replaced by algos, which are way cheaper. In turn, they also require less for upkeep, and so charge less in edge, which means you have better bid/ask prices, lower trading fees, etc.


There are at least 2 benefits:

* They are always willing to trade, so if someone wants to sell/buy a stock, they can get it from an HFT rather than needing to wait for someone who wants it long-term. (The HFT aims to sell it later to someone who wants it, they don't want to keep it).

This is called 'providing liquidity'

* They ensure prices remain 'in sync'. If prices don't match in some sense, then people need to worry about where to get a stock, rather than just what stock to get. The same goes for e.g. keeping stocks in sync with options. This way, someone who buys a stock gets an accurate price.

This is called 'arbitrage'

There is more to these concepts, especially arbitrage. But these are the core benefits of HFT.


It’s been shown by many (see Flashboys) that those “benefits” are completely exaggerated and are just an excuse to do HFT. The only real thing HFT benefits the bank accounts of the traders, and the traders will be the first to admit that.


Flash Boys was pretty inaccurate.

https://scottlocklin.wordpress.com/2014/04/04/michael-lewis-...

Remember that most retail investor orders never hit the exchange because your brokerage firm is selling the orderflow to a HFT firm.

Overall, you can say the benefit is that spreads are cheaper to cross for investors. But, you could also make the counter-argument that if things became slightly more difficult to buy or sell, this inertia would prevent retail investors from over-trading by taking a long-term view.


> It’s been shown by many (see Flashboys) that those “benefits” are completely exaggerated and are just an excuse to do HFT. The only real thing HFT benefits the bank accounts of the traders, and the traders will be the first to admit that.

"Flashboys" is pretty discredited as a marketing exercise for IEX.

Vanguard say HFT lowers their costs (https://www.cnbc.com/2014/04/25/vanguard-chief-defends-high-...). Hardly seems like just "benefitting the bank accounts of the traders".


Flash boys is a pretty bad book that contains downright false information. How do you think that you can get free equity trades on RobinHood? HFT massively helps small investors, while (arguably) hurting bigger institutional players.

HFT is "stealing" from the big guys and giving to the little guys.


Flash boys was a great book for the uninitiated but you have to also take it with a grain of salt as it mostly parrots the point of view of those who got screwed by hft — meaning the ones who either disbelieved the possibility, weren’t imaginative / cunning enough to survive or otherwise thought they could shame the system into accepting the former status quo.


Your first point is literally just market making, which requires absolutely no HFT. Your second point (arbitrage to keep prices in line) also doesn't require HFT and I find it awfully difficult to imagine that (actual) people would need to worry about where to get their stock if arbitrage would occur at less than microsecond frequencies, because otherwise there might be significantly out of line prices at timescales people care about (many orders of magnitude slower).


Market makers need to be able to close arbitrage opportunities or they won't participate in the market. Speed matters just as much to them as it does to the prop traders.


Of course market makers wouldn't want to compete in a market where they are forced to make much slower trades than anyone else. But market making existed long before HFT, so it can't possibly be a unique benefit of it. You can argue that market making has become better because of HFT, but not that it requires HFT.


The technical aspects are more cross-cutting than your labels. "HFT" is for the most part just a latency threshold being crossed, whereas more functional roles such as market making, various forms of "pure" arbitrage, statistical arbitrage, prop trading, etc. have existed at the older latencies as well as the newer. All participants benefit from having lower-latency access than their competition, which creates a (thus far) neverending arms-race as firms leapfrog each other.


The faster a market maker can trade, the smaller spread they can offer. This is clearly a good thing.


HFT contributes to making the market. Making the market contributes to liquidity and the ability to get in and out of things quickly without taking major hits. This seems abstract until you think of cases other than stocks where you desperately rely on a market --

1. Think homes. Consider buying/selling a house and how painful it is to keep something on the market hoping to find a seller (often at a hit if you want to unload quickly.) Or a buyer desiring to purchase something only to have a handful of homes come on the market.

2. Imagine if for every book you read, you could quickly unload the book (or buy new ones) without fire-selling and paying an absurd shipping cost. HFT and market making reduce friction. They are market lubricants.

3. Imagine looking for an apartment to rent and there were a perfect curve of apartments a dollar apart available all the time!


>If we arbitrarily limited transactions to human speed would there be some kind of systemwide downside?

It's worth asking what that artificial limitation would actually be, and how it would result in changing the game for market makers.

Continuous limit order book trading (how essentially all exchanges primarily work) is a fairly straightforward concept, and speed is an obvious differentiator for a successful market maker. In general, when complexity is introduced into trading, that creates more ways to exploit the rules for "unfair" gain. Assuming you believe market makers are necessary for a healthy market, you'd have to think of how some other system (like frequent batched auctions) would change the differentiators for successful market makers, and whether those changes are "good" or "fair."


What if we flip the question:

How fast do trades really need to be?

Lets start with a hypothetical 'human speed' similar to how things used to be in the pre-electric world.

What if there were only one trade made per day? If every day buyers and sellers (with price limits in place) were resolved similar to a dutch auction (or some other 'fair' process).

What if that happened every hour? 30 min, 20 min? 10 min?

I think maybe every 10 min could be a good cycle. That would give a human duration of time for a buy/sell blackout (about 60 seconds), plenty of time for everything submitted to be tabulated, signed, and then published, and 9 full min to figure out what you want to do in the next order period.


This concept already exists: it's a batch auction dark pool.

One big problem with batch auctions as a global solution is that conceptually, trades span multiple markets. You have to figure out how your 500ms futures batch auction is going to interact with the unrelated market for the underlying instrument.

Probably the biggest problem is the problem itself, which, in the case of HFT, probably just isn't enough of a problem to reengineer all of market microstructure to fix. The biggest pure HFT firms are worth just a fraction of the biggest investment firms. Major purchasers of liquidity (pension funds, Vanguard) are happy with the execution they're getting. None of this stuff hurts retail traders (on the contrary, it probably helps them on the whole).


Batch auctions don't eliminate the speed advantage, because it is still incredibly valuable to be the last one to get information into the auction before it crosses. How sure are you that this will decrease whatever problem you think HFT causes?


The best fix to wasteful HFT is actually to increase the precision of prices on the exchange. The reason why these races are a thing is that prices can only change in fixed and sometimes relatively large increments, such as one cent, or perhaps more. Allowing prices to change in smaller increments ought to be good for both increasing liquidity, and removing the incentive for traders (humans or automated) to 'rush' to the market so that their quotes will take precedence over others.


Exchanges do very detailed studies of what tick sizes should be, as well as setting fifo/pro-rata ratios. Many products that have seemingly large tick sizes like eg treasuries or vix would not work otherwise. MMs will not be compensated enough for taking risk and will leave, liquidity will dry up, then liquidity-takers who are unable to execute desired size will also leave.


There were some dark pools that were offering sub-cent pricing. But the SEC fined them and made them stop.


What is "wasteful" about HFT and how would it be reduced with smaller tick sizes? It seems like there would need to be considerably more order activity with smaller tick sizes. I understand the intuition that the ability to compete more on price should reduce the need to compete on speed, but surely it's more complicated than that.

Would love to see some research if it exists.


Presumably, the greater the time period between auctions, the more risk there is for market makers, and therefore the greater cost to customers. I'm sure there's a sweet spot that is less than once per day, and greater than continuous. But perhaps determining that sweet spot isn't worth the added complexity, and letting them fight amongst each other over speed is just better for everyone.


1. HFT market makers creating liquid markets that won’t otherwise exist.

2. Dramatic lowering of bid-offer spreads and, thus, transaction costs to liquidity takers (eg retail investors).

3. No-arbitrage pricing: safe to assume that virtually any easily-tradable instument is fairly priced, otherwise would’ve been quickly arbitraged. Has not always been this way.

4. Driving innovation by paying for technologies/solutions that nobody except HFTs would buy at the time.


It’s most commonly justified in terms of “liquidity” - allowing orders to be filled at as close to “perfect” market prices as possible


I'd think less about the "perfection" of the price and more about the spread. The faster everyone converges on a single understanding of the price, the less you'll have to pay to bridge that uncertainty when you want to buy or sell right now. I always mentally compare our expectations about buying and selling stocks immediately (pennies or less) to that of buying and selling houses "immediately".


From an outsider's point of view (still a CS student):

If you are a CS/Physics/Math person who loves to optimize systems then this is a good way to get paid for doing exactly that I guess. Many other problems/industries just require good enough solutions, with HFT it seems that most optimizations are worth it, which makes it so much more interesting.


All the remaining HFT firms (there's not that many left) have moved into FPGAs and ASICs as well. HFT is interesting because it's not really that finance related at all. The kind of people that work at HFT firms (at least the ones I've met) are exactly people like you're talking about: hardcore CS/EE guys. Some of them don't even care about financial markets at all, they just know how to program some blazing fast C++.

I personally find HFT kind of a snooze and not especially intellectually gratifying. Most of the work is just wading through bullshit (write in interface to some random exchange , figure out how to remove an instruction here or there).

Imo, things aren't looking so good for HFT. The profits are very slim at this point, and staying competitive is becoming exponentially more expensive. The golden years of super-normal profits and fat bonuses are over.



In theory the liquidity created by HFT lowers the cost of capital and increases economic growth. Whether it creates a net benefit relative to the profits it sucks out of the system is unknown. It would be quite difficult to obtain the data necessary to make that calculation, and all of the second-order effects would be complex to model.


I think that a good way to estimate the benefit would be to compare how much less money HFT sucks out of the system than its predecessors.


Lots of other things have also changed since then. It's impossible to isolate the independent variables.


Would this not happen if we did the trades at human speed? Wouldn’t we get to the same result, in theory, perhaps just a little slower?


No, we ran that experiment. In addition to the fact that humans market makers actively colluded with each other to divert money from buyers and sellers into their pockets, the advent of automated trading also decimated spreads. I think maybe the problem is that even if the trades themselves are "slower", the actual underlying activity that the market is pricing isn't, and the gap between the real speed of events and the speed with which the market can price those events expresses itself in uncertainty and higher spreads.


what do you mean by "the profits it sucks out of the system"? How do you "suck the profits out of the system"? First of all, taxes are paid on those profits, secondly profits are reinvested back into the economy.


I learned recently (after getting into day trading) that out of the 6k stocks traded, quite a lot of them are actually very thinly traded. That means it's pretty hard to get into a trade with a reasonable price, because the spread is so big. I guess HFT helps with that.


Good question!

Note that any answers should be HFT specific. That is, if they apply equally well to trades on a second or minute scale as on a millisecond scale, they probably don't answer the question.


Yeah I’d love to know what the effects of limiting trades, to say, one second granularity would be.

I see the importance of market making etc in general, but I don’t see it for millisecond latencies.


What happens is, first, speed is still an issue because life and news do not happen on 1-second intervals. So readers will wait til as late as possible to make sure they have up to date info and then send their orders as late as possible.

Second is the risks to market makers goes up, which means they need to acquire more edge to cover that risk, which in turn widens the bid-ask spread, which makes prices for retail traders worse and will likely reduce liquidity as well.


There should be a 1penny tax on trading of stocks. This nonsense has gotten out of hand.

Many startups look for money though venture capital, and not the stock market, because the stock market is so predatory.

People like Elon Musk have gotten in trouble with the stock market for simple things, like thinking out-loud. I'm pretty sure he hates the stock market, and only by law was he forced to sell stocks.

The stock market has transformed many times. Currently large institutional investors like pension plans buy safe stocks. Originally in the 19th century it mostly a way to buy government bonds. In the 1920s buying stocks was opened up to the 'everyman', a person could buy one share rather cheaply. Then came the depression. In the 1950s the stock-market wasn't that important, economy was doing good. Starting in the 1970s traders started inventing exotic financial tools, to make the stock-market less boring. That's how we got into several crashes, most spectacularly 2008.


Well then you'll be happy to hear that the era of exotic derivatives is mostly over. People nowadays just want to trade standard products (vanilla options, stocks, futures, etc) very quickly.

And the reason startups like VC vs public markets is because they can easily get money on the secondary markets. Going public is a lot of work and comes with a lot of rules. The only reason people did it during the dotcom boom was there was a lot less secondary market capital sloshing around. It wasn't because the stock markets were somehow less "predatory" (whatever that means) back then.


>People like Elon Musk have gotten in trouble with the stock market for simple things, like thinking out-loud. I'm pretty sure he hates the stock market, and only by law was he forced to sell stocks.

Since this is how his company has been kept afloat for the last decade I'm pretty sure he doesn't hate the stock market, he just hates the agencies that regulate it.

You're alone in this thread, but no one here has given any real defense of HFT other than "providing liquidity", which is sheer garbage in a world where we have giant speculative bubbles forming and bursting and taking out huge portions of the economy all the time - it requires a real fanatic devotion to efficient markets to be trying to race against the speed of light. This is bunk; prices are just wrong anyway, and markets don't need to be that good as a result. What we do need is less financialization of the economy. Transaction tax all the way.


I found your last paragraph pretty interesting. Had not thought of this. Thank you.


I don't know when we'll have practical long-distance neutrino transmitters [1], but I'll bet someday the implementation will be bankrolled by HFT firms.

Why go around the Earth when you can go straight through it?

[1] https://phys.org/news/2012-03-wireless-message-neutrinos.htm...


Fermilab is literally a couple of miles from the CME datacentre in Aurora. They must have some old accelerators they aren't using.

The original Kamiokande detector is still in the ground at the Kamioka Observatory, as far as i know. Hasn't been used for science since the '80s. That's about 300 km from either Tokyo or Osaka, so you probably still need microwaves at that end.

Now you just have to think of a trade with a signal in Chicago and an instrument in Japan. The CME has Nikkei futures, but they follow the ones in Osaka. Osaka has Dow Jones futures, which presumably follow the ones in Chicago, but the volume in Osaka is pretty small. Your best best is probably FX - both Chicago and Tokyo have dollar/yen futures with good volume.

So, who is already doing this, and how long have they been doing it?


The first message sent by telegraph was supposedly "what hath God wrought?"

The first message sent by gravity wave will be "SELL 400 AMZN"


I wonder what's stopping them. How sensitive does a neutrino receiver need to be to become viable?

One thing I am most excited about is the advent of low throughput ( ~100 bps) communication with the rise of neutrino communication. The optimizations needed to use low throughput link would be interesting to see.


I think when we get into cool near sci fi stuff we'll have faster than light communication pretty soon with quantum entanglement effects, which should be way easier to wrangle than neutrinos.


Quantum entanglement cannot transmit information faster than light unless contemporary physics is very very wrong.

Neutrinos may be hard to use but they at least known to be physically possible.


I work in HFT as a quant dev

I am proud of my work and find HFT to be an interesting intersection of computer science, math, game theory, economics, and psychology

As far as the societal benefit of HFT, I think that it is neutral. It doesn't hurt anybody. It might even help a little bit (liquidity and tight spreads save people a little money and maybe spur some economic growth).

However, this particular aspect (the physical latency war) makes me literally sick to my stomach. I hate reading about it and I hate thinking about it. It's like -- what are we even doing here guys?


It's only neutral (at best) if you don't consider opportunity cost. Is the best way for the country/society/humanity to deploy a bunch of smart people with a decade and a half of education truly an attempt to try to cheat the speed of light? If trades took 2x as long to execute as they currently do, what exactly would be the harm?


Even supposing that HFT adds no value to society (which is an incorrect assumption as many commenters have already pointed out), this particular argument is pretty inane in my opinion.

There are perhaps 4-8 significant players in the HFT space, all employing anywhere between a few hundred to a thousand people. Probably less than 5000 people total, and if you only include programmers and quants, maybe less than 2000. Google alone has 30000 programmers. How many of them are optimizing ads for eyeballs vs “benefiting the country/society/humanity”? What about the thousands of programmers making video games mechanics more addictive for children? I think it’s silly to focus on one tiny industry, and demand that those people should be working on things for the good of humanity, when the fact is the vast majority of humans just work for their own livelihood. I mean, what do you think 95% of the people in SV are doing? Not every company is trying to cure cancer or explore space...

HFT is such a small, niche, industry, and also extremely productive per employee relative to most other industries. (the primary reason for comp being so higher). It replaced the thousands of manual traders that used to be responsible for arbitrage and market making with automated robots, dramatically increasing market efficiencies while reducing the amount of human capital required to provide those services. I find all the virterol towards hft from SV very puzzling, as it seems to exemplify all the things that SV espouses!


Well, you would have lost out on a lot of innovation for one. I would be willing to bet that nearly half of the world's high performance computing specialists cut their teeth with a trading firm at one point or another. By your logic, there would have been no benefit to video games or going to the moon even though each of these things have helped deliver large amounts of technological advancement.

IE, Facebook is a giant waste of resources but they employ many of the world's smartest engineers. Because of that they've spawned things like ReactJS which has completely changed web development for thousands of other companies.


The "brain drain" argument is fair, but you can successfully apply the brain drain argument to anything short of curing diseases or feeding the hungry

I consider HFT to be no worse of a brain drain than, say, Instagram, or any similar virus


> It's only neutral (at best) if you don't consider opportunity cost.

I might be missing something, but isn't opportunity cost a good argument for HFT?

HFT employs far fewer people than pre-HFT trading that came before it.


People are not "deployed" into HFT, they exercise their free will. Their livelihood is not your opportunity cost.


Also advertising....


> It doesn't hurt anybody

Not anybody in particular but it does hurt society as a whole. It's one of those things that make the rich richer for no reason other than them already being rich.


This is fair to say, but HFT is a very small industry. The number of people getting "rich" off of HFT is less than 100. The rest of us are just comfortably compensated tech workers, same as a Google or FB engineer


Liquidity dries up very very quickly when the shit hits the fan though. And HFT is the first to stop buying.


And they’re the first to start buying as well.

You want to see what happens without HFT. Look at the crash in 89, when human market makers would no longer answer their phones because they did not know what was going on in the market.

Instead of the crash recovering in minutes, it took hours and had larger widespread effects.

Furthermore, exchanges have now added automatic pauses in trading during highly volatile times, to let traders figure out what is going on.

The markets are objectively more stable, more efficient, and tighter, and better for the general public now with HFTs than they were in the 80s and 90s with human market makers/brokers.


Liquidity drying up when shit hits the fan is a fundamental aspect of markets that has been around forever

I won't go as far as to say that it's a feature not a bug. But I will say that the problem is currently the least problematic that it has ever been. Because of HFT, stability returns within minutes, rather than hours/days/weeks (which is how long it used to take before various technological innovations)


When shit is hitting the fan, participants become willing to cross _very_ large spreads to trade. It can be a lucrative time for HFT market makers, as well.


How can you possibly say HFT is neutral when it's been responsible for multiple flash crashes that hurt the average investor?

Not to mention the practice of front-running and other shady practices that HFT's use to make profit.


> How can you possibly say HFT is neutral when it's been responsible for multiple flash crashes that hurt the average investor?

The average investor holds onto stocks for months to years. A flash crash may last minutes, before the price is back where it was. How does a flash crash hurt the average investor?

> Not to mention the practice of front-running and other shady practices that HFT's use to make profit.

Front running is where a broker or other agent trades on private information given to them by their client: the broker receives a big buy order from their client and then buys the market up on their own account before executing the client order and making a massive profit. This is illegal.

HFTs can't front run - they only have the same public information at the same time as everyone else. The fact they can execute on that faster than others hardly seems 'shady'.


> How does a flash crash hurt the average investor?

Maybe it can trigger "stop loss" orders?


Maybe I'm biased, but I say that anyone who uses stops has voluntarily left the "average investor" realm and entered the "amateur day trader" arena


> > How does a flash crash hurt the average investor?

> Maybe it can trigger "stop loss" orders?

The average investor is a person who has a pension invested across a range of funds held with large fund management companies. Anyone mis-using stoplosses is not an 'average' investor.


Running stops was a common pit trader game.


HFT firms perform a different kind of front running. If I'm an HFT firm and I see a large buy order on the books at exchange A, I can use my low-latency links to exchanges B, C to execute my own buy orders there before the rest of the market reacts.

It's not illegal, but it is still front-running the rest of the market due to latency arbitrage. Check out Flash Boys by Michael Lewis for more on the topic.


This isn't a definition of front running that anyone in the industry uses.

As you say, this is just arbitrage on public information. Latency arbitrage has been occurring for the last 200 years at least, it's an inevitable outcome of any system with multiple, geographicly separated markets.

P.S. I read Flash Boys. It's not very good.


This hypothetical HFT firm is part of "the rest of the market reacting"

They just happen to be the first among all participants to react. Why is that bad? Somebody has to be first


Perhaps I have misunderstood what front running is, but I have always thought it is really only applicable when you have an obligation to your clients to give them good execution. You can't front run a market, just your clients. An HFT has no clients on behalf of which they are executing trades, they are just trading for their own account.


Location is everything. If trading firms are willing to spend money to position their towers slightly closer to the trading endpoint, the town where the endpoint is located should approve construction in geographic order, allowing new towers to be constructed closer to the endpoint in a way that maximizes tax revenue to the city.

Ideally this would keep a lot of tower-construction people continually employed and paying taxes, building new towers ten meters in front of the old ones every month or two.


Fun fact: New Line Networks mentioned in the article is named after the New Line Tavern in Chicago [0]. Which is actually quite good. And might just attract network engineers that work for HFT firms. Purely coincidentally.

[0] https://www.newlinetavern.com


New Line Tavern is the only bar I've seen that has drink/food specials on Friday and Saturday.


This film seems relevant - https://www.imdb.com/title/tt6866224 - The Hummingbird Project (2018).

'A pair of high-frequency traders go up against their old boss in an effort to make millions in a fiber-optic cable deal.' (It also seems to feature the microwave based approach from the trailer).


I wonder if a lot of money is going into research on "spooky action at a distance".


I get that arbitrage helps set the price, but what does HFT do, beyond deciding which of the competing nano-second enabled traders get to edge a profit in an artifact of how information flows?

I don't see this HFT as being information-worthy arbitrage. I have an intent to set a real-world price, I declare it, and in the increment of time it takes my packets to flow, somebody edges in the queue and inserts trades to make me have to deal with them for some increment of price. I wind up paying more, and the original source winds up getting less, or some other event.

its like re-intermediation, in a world heading to dis-intermediation.

What is the real-world, wider economic "good" here? If my pension fund is routinely profiting from this, I am not sure that is a wider good btw: We could probably make more money selling Heroin, if we're going down this 'whatever it takes' track...


> it takes my packets to flow, somebody edges in the queue and inserts trades to make me have to deal with them for some increment of price

This is the biggest mistake people make when talking about HFT. At no point in time does someone see your order before it hits the exchange (unless you’ve specifically agreed to trade that way & even then they are obligated by lots of regulation to execute in particular ways to protect from that agency problem).

No HFT is taking pennies from your order. If you send a limit order through the book you will get filled at the best price available up to your limit.


Thanks for explaining. If they're just squeezing margin by being quicker, and not intruding into my trade (I don't trade. this is clear I think, given how ignorant I am) then I have less to worry about.


I don't think anyone asserted there was a real world wider good; to the contrary, I think it's just the actions of a system in place.


Oof, solid article but Bloomberg breaks the back button if you scroll past the end of the article. It will load several new articles and push them onto the nav stack, meanwhile it doesn't seem like I can click enough times to get back to HN from there.


Are you on mobile?

The article reads fine w/o JavaScript, desktop Firefox. And the back button keeps working. I assume that's thanks to the absence of JavaScript, rather than something special that Firefox is doing.


Another example for people to appreciate what a small latency advantage means. Consider SPX options which is a huge market. These “trade on CBOE” which is a Chicago exchange - but they physically reside in NJ. Now consider SPX futures - ES - on CME, physically in Aurora. Now imagine a 20-tick / 5-handle “sweep” in the ES — “sweep” is instantaneous price move (either up or down). Not “very fast” but exactly that - instantaneous. These happen as a result of large incoming aggressor order taking out many book levels at once. Sweeps can be of different magnitude (from just few ticks all the way to max allowed per CME’s “velocity logic”) - but they happen all the time. If you know enough about e.g. ES futures but have never seen sweeps, chances are you didn’t have access to the right tools/platforms. Even some supposedly-professional-grade platforms like TT or CQG do not show trade data with enough detailization to see these.

Anyway... why is this important? Because immediately following the sweep in e.g. ES, option prices in e.g. SPX options are temporarily way off. Dutch market makers have not yet moved their options quotes, because they do not yet know about the sweep - at least not in NJ. If you can somehow get orders to NJ faster than market makers can cancel their option quotes, you can take out hundreds of mispriced options. Multiple expires, dozens of strike, puts and calls. Very juicy bunch of stale quotes.

PnL... well that gets tricky. On one hand, just this one example is in theory worth hundreds of million per year if market did not respond. But the market will respond. That’s inevitable. No one enjoys being ripped off for too long, so if they cannot defend they will either leave the market or set quotes wide enough that they are never mispriced. The same goes for any other “latency arbitrage” game. I always love how people throw numbers like “this is worth X”, almost always assuming status quo. HFTs are very responsive to market conditions. If I run a strategy that’s been making money every single day for 10 years and all of a sudden it starts losing - I press the pause button and go do some serious analysis. So maybe I will let you have it for few hours - but after that if I can’t compete I’m out. So it’s not that simple.


The dice is fully loaded in favour of HFT pioneers and incumbents, facilitated by the ever-growing and nouveau riche (e.g. Zayo) Tier-1 ISP's, who control most of the backhaul and dark fibre estate.

However, over the next few years, the advent of maturing ML frameworks, coupled with the intensifying war and ratification of standards over 5G/IoT based infrastructure, might prove to be serendipitous for nimble-footed enterprises.

https://en.wikipedia.org/wiki/Zayo_Group


Highly unlikely that 5g / iOS will play a role in the context of hft. Also your view of the ecosystem (eg zayo and co being the big winners) is massively outdated by five years at minimum.


The use of wireless communication in high frequency trading begs the question: how much could a malicious actor profit from momentarily jamming that CyrusOne (or similar) tower?


I'm largely ignorant of HFT but I often wonder what the benefit of it is. It sounds like a huge zero-sum game to me.

In my (probably simplistic) view, exchanges could publish and process orders at long intervals (seconds, minutes, hours?) then add a jitter of several intervals to incoming orders, with cumulating jitter for cancels. That would require traders to look beyond one interval when trading.


Traders already look beyond one interval. An order I put in yesterday that took 5 minutes to execute still beats an order I put in today in 10 nanoseconds.

Being fast is important for cancels & risk management. Which answers the question on what it provides. It lowers the risk for the traders allowing them to price tighter.


But lowering the risk can only happen at the expense of other traders. So overall there is no lowering of risk. HFT is just increasing the cost of market participation for other traders. Because they pay the profits of the HFT traders.


No. We can objectively show that is untrue. The risk for HFT (especially market makers who are most sensitive to latency) is priced via the bid/ask spread. That is smaller than it’s ever been and would be smaller still on many symbols if minimum price of ticks was lowered.


I don't understand why I should care about the risk to HFT traders because I don't see why they should exist in the first place. So I don't understand your argument.

For HFT traders to make a profit, somebody else must lose. In my understanding, the ones to lose are the other traders. If I want to buy shares, part of the price of buying them will go to HFT traders. And my revenue of selling them will be lower due to HFT traders on the exchange.


One of the biggest classes of "HFT" is automated market making. These participants have long existed in trading environments and are there to bridge the gap in time (you want to buy now, but no one wants to sell) and space (you buy on exchange a, but the participant that wants to sell is on exchange b).

Market makers take on the risk bridging those 2 areas. They hold positions for short periods, with the hope of making money when a new participant enters the market. They lose money when they are holding positions that the market goes against. To account for that, they price their risk into what they are offering you. The less risk they take on, the better price they can offer you.

If you want to buy shares _right now_ you are going to pay someone for the privilege. Modern HFT are offering that service cheaper than any other implementation before.


Ok thanks for the explanation. I still don't see what benefit I could get from buying shares right now as opposed to a few seconds later. Why should I pay for that?


If you don't cross the book (ie offer more than the lowest bid or vice versa) the you _become_ a market maker (though not much of one). You don't pay anyone to do this. But because of the HFT/Market Makers you get the benefit of dramatically lower trading fees.

So you don't pay for it in that case, but you still get a benefit.


would you rather trade on exchange that adds jitter to your order, or one that doesn't?


Depends on my goals. If I'm buying or selling for investment, I just don't care about that jitter. Because my decision to buy or sell doesn't happen at that speed.

If I'm racing other people's orders on the sub-second level then I care. But what's the point of that? Why should people spend their lifes on it? Why should we as a society bear the cost of this activity? I'd rather see people do something with external benefits.


What is the point of adding X ms delay to your order? What does it fix? The "problem" with HFT is that participants can quickly pull out the limit orders, so your market order will be executed at the price that is worse than you expect. Jitter doesn't fix it. If anything, it will make it worse, i.e. your order will hit the market that is quite different from what you saw when you sent the order.


Not delay, jitter. It removes the possibility of timing orders to sub-second intervals. It's supposed to prevent people from spending resources on a zero-sum game.


I did the math to use the neutrino transmitter it'd be 0.13% faster to transmit information from Chicago to NYC. Using equations:

s = r(theta)

l = (2r)sin(thea/2)

s: arc length (air travel distance 714 miles)

r: radius of earth (3,958.8 miles)

theta: arc angle

l: length of cord (straight line under arc)

(s-l)/l * 100 gives the % increased difference in length


You need to add a little extra time to the microwave route because microwaves are line of sight. The signal needs to be detected and re-amplified multiple times along the way. How much extra time? I don't know, I'm not an RF guy, maybe someone can chime in and tell us how much latency each amplifier adds.

The neutrino route only needs a single detector at the very end, nothing along the way.


If you want to know how HFT works I recommend the book Flash Boys by Michael Lews or the first 15 minutes of this talk by Brad Katsuyama (the book protagonist) https://www.youtube.com/watch?v=N9hoqFpDjVs.

In short, when you send an order from your computer to buy say 10000 stocks, this order gets split to all the multiple exchanges. When the order hits the first (closest) exchange, HF Traders will pick that up and race you to the rest of the exchanges and try to front run you by buying the stock in lower price if available and make an offer to your bid price that they know is coming.


Flash Boys is terrible (it's not even good as a book --- and I'm an avid fan of Michael Lewis's). It's embarrassingly bad on the technical details.

I like "Flash Boys: Not So Fast" as a corrective.

A much better book, from another HFT skeptic, is "Dark Pools".

The example you provide (of basic, naive latency arbitrage) matters for trading desks at giant investment banks. They make their money in part by selling execution services to giant holders of stock. Major trades by these people (big "blocks" being "shopped") are tradable market news. The giant holders of stock would prefer it if the news didn't get out until after their orders were filled. That's a service people like Katsuyama used to be able to offer his wealthy clients with very little effort on his part. That service has since been automated away, and it bothers him. I guess I'd be bothered too, but I'm not sure why it's moral dilemma.

Front running isn't a technical problem and it has little to do with how fast traders are. It's an agency problem: a broker "front-runs" their client when they receive an order from that client and make proprietary trades against and ahead of it. Random people in the markets don't have an agency relationship with other traders and are under no obligation to avoid competing with them.


I really enjoyed Dark Pools. The asm + FoxPro source code for the Island ECN matcher described in the book has been released on the net and that amused me greatly when I was working on an order matching engine a few years ago.

I love exchange technology, it is the beating heart of capitalism and I enjoyed working as a plumber in the machine.

Josh Levine is an inspiring idealist.


I was an engineer for a high frequency trading desk at a firm not mentioned in this article.

Flash Boys is a great book and we required our new hires to read it as it got a lot of things right.

Front running is illegal now, but arbitrage opportunities still exist between exchanges. Being fast is still an easy way to make money.

You're right about dark pools though. Those are the future (honestly they are already up and functioning) and they hide trading activity very well.


This is kind of a series of non-sequiturs.

Front running has always been illegal.

Arbitrage is not illegal. Latency arb certainly isn't easy.

"Dark Pools" is a book. It's also a thing, but that's obviously not what I was talking about.

There's nothing especially shady about dark pools. We'd call them "private exchanges" if the exchanges weren't themselves already private companies, I think.


Front running has not always been illegal, front-running if you have a fiduciary duty to clients has always been illegal, and many forms of front running are still legal today.

Nowhere did I imply that arbitrage is illegal and latency arbitrage is a big driver of revenue for several hedge funds, especially those which track baskets against their components.

There isn't anything shady about dark pools, except that there is little public information about the trades which take place within them, they are sparsely regulated, and you have even less insight into your counter parties than you do on open exchanges. And my response referencing dark pools was a response to your comment about execution services.


Front-running has been illegal (under that name) since 1987, long before the full automation of trading. Again: front-running involves an agency relationship; by law, you're not "front running" simply by beating a part of someone's block trade to an exchange.


Are you sure you worked at an HFT firm? Front running has always been illegal as it is basically insider trading.

Being fast is not an easy way to make money. It's an incredibly hard and difficult way to make money. I don't know what kind of HFT you worked at, but I guess you just plugged in a laptop to a GBit/s network line and plugged away making "easy" money?

Also, the first dark pool was created in the 80s, so you're a little late on that development as well.


Uh, being fast is not easy nor cheap.

And once you are fast, you have to stay fast. Remaining fast is not easy or cheap either.

Sure, if you have the part that’s really hard to get right, making money is then really easy.


Note, me & every other person I knew in the industry were extremely disappointed with Flash Boys. It was a submarine ad for IEX & got things just plain wrong.

Feel free to read it but remember that IEX & Brad Katsuyama work on behalf of one set of market participants that are decidedly against the interests of other market participants such as retail traders.


> In short, when you send an order from your computer to buy say 10000 stocks, this order gets split to all the multiple exchanges. When the order hits the first (closest) exchange, HF Traders will pick that up and race you to the rest of the exchanges and try to front run you by buying the stock in lower price if available and make an offer to your bid price that they know is coming.

1. This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different.

2. Orders don't get split across multiple venues by accident. It's a conscious decision. Anyone who chooses to route multiple orders to different venues in a way that causes an information imbalance (ie, so the orders hit the venues at different times) is going to get their lunch eaten by the rest of the market. Only idiots do this - the rest of the world times their orders taking transmission latency into account.


> This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different.

I guess that depends on what you consider public market data.

> Front running, also known as tailgating, is the prohibited practice of entering into an equity (stock) trade on advance, nonpublic knowledge of a large pending transaction that will influence the price of the underlying security

When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors.

It seems to me that the owners of those feeds have advanced knowledge of non public data.


> When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors.

This specific behavior was only exhibited by DirectEdge (I don't recall if it was a registered exchange at the time, or just an ECN). The behavior was indeed scandalous because of the overt pay-to-play overtone, and was eliminated fairly quickly. Lewis somehow managed to mischaracterize all on-exhange HFT as this type of trading.


> > This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different.

> I guess that depends on what you consider public market data.

Usefully, the regulators define it for us. No consideration needed.

> When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors.

> It seems to me that the owners of those feeds have advanced knowledge of non public data.

Can anyone buy one of these feeds? If so, it's public data.

"Non public" means something very specific (and different) in this context.


Semantics aside, the gist of the issue is: should exchanges offer faster data feeds as a service? In a perfect world the goal of an exchange should be to facilitate fair trading not to make money by allowing sharks to feed on the rest of the participants.


You think those guys in colorful coats sitting in the pits just walked in and started trading for free?

The exchanges have always had participants with more access to faster info than other participants. Electronic feeds & order gateways are orders of magnitude less expensive than the previous open outcry regime.


anyone who wants the "faster feeds" are welcome to pay for it. hfts being able to trade on better infrastructure is no different from walmart being able to distribute soda to you at a fraction of the price they are retailing it at.


> the rest of the world times their orders taking transmission latency into account.

Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be?


> > the rest of the world times their orders taking transmission latency into account.

> Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be?

Simplistically, If you're making an order that's too big for the order book on a single venue to get a sensible price, you might chop it up and buy some in NY, some in London and some in Tokyo. Let's say you're sat in NY: you can work out that it'll take your order 1ms to hit the NYSE, 150ms to hit the LSE and 600ms to hit the TSE. So rather than dumbly hitting 'transmit' on all your orders at once, you send the Tokyo order first, then then the London order 450ms later, and then the NY order 149ms after that. That way, your orders (and the public information that your making those orders) will hit each venue at exactly the same time.

This is common practice for any broker or institutional trader.


Follow @sniperinmahwah if he is still active... everything else is stuff that’s been known for quite some time.


Could they not just make one tower a few feet higher than the other?


Why don't the rich people just move to New York?


Capitalism is just so absurd.




Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: