Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
HFT Quote Stuffing is a software bug (chrisstucchio.com)
62 points by jpdus on April 22, 2014 | hide | past | favorite | 63 comments


So again with all HFT conversations we have to be clear on what we mean with the language we are using.

I've seen quote stuffing used to mean what Chris mentions here. The implication being that HFTs were using large volumes of low quantity orders for nefarious purposes such as DoS the exchange & SIP or confusing your competitors by making the order flow to heavy to process. These kinds of games did occur, but it was very briefly a long time ago because the exchanges themselves had a vested interest in stopping it and an easy mechanism which to do so (fill ratios). I prefer the term quote spamming for this, and I agree with Chris that most of the time when this happens it's just a bad order management bug.

Quote stuffing can also refer to the practice of putting a few large orders into the order book with no intention of trading them in order to make the volume on the book look larger than it is, and then cancelling them quickly to take advantage of the "real" volume. This is a predatory activity designed explicitly to take advantage of market making strategies and is already illegal. Your mileage may vary when it comes to enforcement...


The second kind of quote-stuffing you mention, where you place larger orders outside the best bid/offer with the intention of distorting the order book (aka spoofing, see comment below) is certainly nefarious. But it's a special kind of nefarious behaviour that really only affects other market participants who can react quickly enough to the information - i.e. it's mostly HFTs taking money from other HFTs.


I think it is more short-term day traders taking money from naive HFTs. I'm sure some people automate this type of manipulation, but I seriously doubt that any of the "big name" market making firms do it. They are under a lot of regulatory scrutiny, make more than enough through legitimate trading, and have the technology & data to enable their internal compliance auditors to detect it.

I also don't think it can make that much. It may work for a little while, but no algo is going to sit there bleeding money to you forever without hitting a risk limit or being reviewed/adjusted. If your business model depends on somebody else doing something dumb, it's not going to work for the long-haul.


The cases I know of where the SEC has cracked down were all manual traders leaning the book and essentially taking advantage of naive MM algos. Proceeds < $5mm each time I believe.


But it's a special kind of nefarious behaviour that really only affects other market participants who can react quickly enough to the information - i.e. it's mostly HFTs taking money from other HFTs.

This kind of behaviour that undermines the integrity of markets should not get a pass based on who it does or does not hurt, tho. If nothing else its bad PR that keeps people out of the markets or creates other "absences" or dark patterns from people not showing up to trade, fro whatever reason.


Using large non-bonafide orders to lean the book in the direction you want to trade is called 'Spoofing', not quote stuffing. It's been illegal for a long time.


I agree that some people call it spoofing and that is a better name for it, but other people do call it quote stuffing.

It's one of my pet peeves that when it comes to this industry there is very little standardization of the language.


Misuse of language by opponents of automated trading is intentional. When presented with examples of beneficial computerized trading like automated market making, they throw their hands in the air and say, "oh no, when I say 'HFT' I only mean these bad kinds":

"The practice of spoofing, or sending in fake orders in order to gain information about what other investors, traders and other algorithms are doing, is the corner stone of most High Frequency Trading strategies (HFT)."

http://www.nanex.net/aqck2/4371.html


I always assumed that step 2 was "DoS your competitors".

This is crudely done with volume of traffic, but more elegantly done with frequency and amplitude (and so on) given some set of (possibly wrong) assumptions about how competing algorithms will respond to this rapid fire order placement.

I know almost nothing (beyond ZH and chapters 1-5, so far, of _Dark Pools_) about HFT, but I can trivially think of ways you could be evil with quote stuffing as it is described in the OP.


Manipulation definitely happens where traders will show non-bona fide interest on one side of the market to provoke other traders or algos to react. It's known as layering or spoofing and like any behavior where you're placing orders to influence others rather than out of an interest to trade, it's illegal.

Most cases of this or similar behavior like stop running have involved human traders at loosely regulated click trading shops or smaller electronic firms that don't play by the rules:

https://www.finra.org/Newsroom/NewsReleases/2012/P178687

http://www.cftc.gov/PressRoom/PressReleases/pr6649-13

Aside from the regulatory risk, it doesn't fit the business model for most "HFT" firms, who generally are making many low-risk, low-reward trades with a small statistical edge based on proprietary models. Spoofing usually requires showing large false interest which entails considerable risk if you actually get elected. If you are fast enough to make markets or arb crude oil, you can make way more doing that than manipulating the DOM hoping to push algos into doing something stupid until they figure your tricks out.

Predicting markets is hard. Predicting how thousands of heterogenous agents will react to your order is even harder, if not impossible.


"Predicting markets is hard. Predicting how thousands of heterogenous agents will react to your order is even harder, if not impossible."

My very small amount of knowledge on this topic leads me to believe that the idea that one algo would specifically target another algo is not at all farfetched. Who cares how the market responds to your weird trading if you're trying to create distortions for one, or a few, actors ?


Because the market has many actors. If I try to push the market down to shake out stop losses, another trader may say, "Hey, this price move doesn't make sense considering recent activity, how the broad market is moving, etc., I'm going to buy more" and prevent me from doing so. Likewise if I'm shifting the depth with a big false order. I might induce some actors to do bad trades, but another actor might see my order as a great opportunity to trade in size and trade against it, so I'll take a loss.

Liquid markets with a diverse set of actors are more resilient to manipulation attempts. It's hard to do unless you have more capital or are willing to take risk that other actors in aggregate are not.


'"Hey, this price move doesn't make sense considering recent activity, how the broad market is moving, etc., I'm going to buy more"'

Betting you know more than the guy moving large size rarely a good move, though...


So since all orders have to go to the exchange first, how does DoS your competitors work?


You're thinking of a crude, direct DoS attack wherein the "traffic" generated by one HFT directly overwhelms another HFT.

But there are other ways to DoS - what if I observe your trading activity and I discern something about your underlying algorithms and how expensive those algorithms are in terms of memory or CPU or latency, and I make a bet that a certain trade pattern will dramatically overwhelm your resources via these second and third order effects...

It's just speculation, but the point of the OP was that there was no way this could be useful to an HFT algo and I am pointing out that in a few minutes we can come up with all kinds of ways that it could be.


Uh, what if I observe your trading activity

Since the traffic is anonymous, how do you propose to do this?

[Edit] one HFT directly overwhelms another HFT

No, the dos needs to go through the exchange, thus is is not "directly" but indirectly through the exchange.


"Since the traffic is anonymous, how do you propose to do this?"

Traffic other than your own is anonymous, but that doesn't mean you can't spot patterns that are almost certainly the behaviour of one other agent. On some exchanges you can track the history of resting orders, too, which gives you some more info (though still doesn't tie all of any player's orders together, anywhere).


http://zacharydavid.com/2014/04/on-hft-part-ii-bugs-features...

This blog post details similar issues that can lead to rapidly canceled orders. Generally, firms that are smart avoid moving their prices around "too much" since:

a.) You lose your spot in the queue. If I am near the front of the price/time queue buying at 100.01 and move to 100.00 and back up again, now I'm behind everyone else and will either lose opportunity or face more negative selection (I'll only trade when someone wants to sell all the contracts bid at 100.01, whereas before I would get some trades where someone only wants to sell a few).

b.) Sending orders is more computationally expensive than doing nothing. If you send enough orders you will slow down your own system.

c.) Depending on market rules and behavior the messages you send may cause your gateway to queue up or block you from sending more messages which hurts you.

I don't think quote flicker can provide any "evil" benefit to the person doing it. It doesn't make sense to do it to slow down competitors, since most feeds are multicasted to everyone and you still have to process all the orders you send, both from the gateway and over the data feed. If you are faster at handling this situation than your competition then you are going to beat them anyway so what's the benefit in slowing them down?

I do think it can be disruptive in the sense that it places extra load on the exchange and can annoy participants who have trouble taking prices on their screen. Some markets have quote/trade rules to encourage more efficient messaging which is their right and not a bad idea if the limits are set reasonably. Rules like this make more sense than a minimum resting time, which just penalizes makers at the expense of fast takers. Imagine I am putting quotes out in SPY, an S&P 500 ETF, but they need to rest for half a second. Someone sees the S&P 500 futures or the individual stocks move and then my quote becomes a free option to trade at an unfair price. In a market like this, market makers would be forced to put up wider prices to compensate for these losses.

A high quote/trade ratio in itself is not disruptive. A lot of the Nanex charts use thinly traded derivative products like ETFs as their whipping boy. The underlying value of these products will change whenever another stock or commodity changes, so market makers need to update their quotes quite rapidly to avoid trading at unfair prices, even if no trades have occurred in the product itself.


> If you are faster at handling this situation than your competition then you are going to beat them anyway so what's the benefit in slowing them down?

Suppose your fund has invested a lot in some hefty compute hardware but your trading strategies are on the weak side, whereas you know your competitors have better strategies but their computers are slower. Isn't it then in your interest to stuff the order feed and make everyone more dependent on computer power?


Maybe? Most guys who are really fast are also really smart. It's hard to make money purely being fast these days.

I can't speak for everyone but most "HFT" models are built around providing some kind of service to the market, generally either making markets to try to make the bid-offer spread or arbitraging away inefficiencies between related products.

Even if you are evil, building a model that solely seeks to take advantage of another trader doesn't make sense in the long run. Eventually they will alter their behavior or exit the market. It's not sustainable.


Whatever action you take will generally come back down the multicast feeds back to your own systems to process.


Nanex linked an interesting CS research piece[1] which also discusses other possible reasons for (malicious) quote stuffing including:

- walking others into the book,

- creating false midpoints and

- trying to cause stale pricing and slowing others down.

[1]http://de.scribd.com/doc/116761218/CS-HFT-DETECTION


But even they admit that a market flashing between 100.01 offer and 100.00 offer is sometimes beneficial for them. It says their execution algo will attempt to "pick off" the 100.00 offer with IOCs if they are representing a buyer.

Flashing orders with the intent to manipulate is wrong and is illegal already. It doesn't matter if you do this by hand or with a computer. Putting up a tighter best price, even if it doesn't stay put very long, can only be beneficial. At least some of the time, other traders will get filled at a more advantageous price than they would otherwise.


That sure is a lot of hand-wringing for what is a fairly simple and obvious problem. You're reacting to the extremely short-term derivative of a noisy signal. Of course things are going to go crazy from time to time.

It's odd that he touches on the obvious solution of rate limiting, but doesn't actually discuss it as a way to stop this problem. If quote stuffing is a bug that harms the people carrying it out, then why don't HFT programs have a rate limiter on them that make them stop? He even goes so far as to say that you can get shut down for doing this, so why isn't there a tripwire to make sure you can never, ever do it?


If quote stuffing is a bug that harms the people carrying it out, then why don't HFT programs have a rate limiter on them that make them stop?

They do. But a rate limiter is just another signal - you still need to set the rate limit high enough that you can correctly respond to market conditions, but low enough that you don't go wild.

"Never, ever do it" is like "zero tolerance for bugs". It's a goal to strive for, but it's not achievable.


It's completely achievable. Rate limit to one trade per second. Done.

You don't want to do that, because it doesn't let you make as much money as you otherwise could? I totally understand. But in that case, quote stuffing is not a bug, it's a deliberate tradeoff.

But I suppose that "we let this happen on purpose because it allows higher profits, even though this particular behavior doesn't help us" doesn't sound as nice.


The point I'm making is that it's undesirable behavior that doesn't directly help the HFT. If they could stop it without going to ridiculous lengths like your proposed "give up trading entirely", they would.

And given that quote stuffing doesn't actually hurt anyone, it's not like stopping it is a high priority.


>And given that quote stuffing doesn't actually hurt anyone, it's not like stopping it is a high priority.

It does hurt people in some way. It increases the amount of data we need to handle. More storage space and more processing required.

And you've been a bit disingenuous in your article - there is a theory behind the "2.???" part. The theory is that slower participants are overwhelmed by the quote traffic and end up lagging, and are then taken advantage of. I don't personally believe it, and I'm sure 99% of these things are just feedback loops, but it is theoretically possible. The faster exchanges are cycling in say 20-30 micros these days, so any participant who can't match that rate will easily fall behind. Who knows, weaker participants may still be taking 1-2 millis to handle an update. So just before some known vol event, say an economic figure, you flood the exchange and lag the other participants by say 50 millis, then pick them off after the new information arrives.


There are traders A and B in addition to yourself. A is as fast as you, B is slower.

How does it help you to slow yourself and C down? That just means A will beat both of you.


You send your burst a few milliseconds before an event. It chokes up C (all of those depth updates sit in his queue and he's busy for the next 50 millis churning through them). Then both you and A are free to pick him off.

The thing is, you know to ignore the burst on the public feed, since you know it is coming (you created it). You still have to process the order book updates, but you can skip your decision logic. So any participant who has an internal latency (packet decode + orderbook update + decision logic) greater than 1 exchange cycle, and who doesn't have an "out" clause to stop making decisions if it is falling behind, will be vulnerable.

Like I said, I don't buy it, but it is possible in theory.


What prevents A from "picking me off" (which I assume means "get to the top of the queue")?

Also, "know to ignore the burst" doesn't mean you don't have to process it. It merely means you need to subtract the result of the burst from your signals.


If there are any weaker participants may still be taking 1-2 millis the technical term for this is "non-participant".


Well, the point the article is making is that quote stuffing is a bug, but it's not. It's a side effect, but one that's allowed on purpose, because it's necessary to do the things they want to do.

I have no strong opinion about HFT. I don't have a lot of interest in the stock market in general. I do, however, have a low tolerance for weaselly language and misdirection.


It's a side effect, but one that's allowed on purpose...

Similarly, if you build a system on MySQL and later see dates like `0000-00-00` (yes, really), that's not a bug. That's a "side effect, but one that's allowed on purpose". Pretty much all bugs fall into that category - after all, you could have spent more time on formal verification/testing or just wrote the whole thing in Coq.

I'm sorry you found the language "weaselly". My sole intent here was explaining the mechanics behind quote stuffing. How would you prefer I phrased it?


I'm afraid I'm not familiar enough with MySQL to understand the analogy you're making there.

As for wording, don't call it a bug. Say that it's a necessary side effect of making money with HFT. It's an inherent design limitation. The word "bug" implies that it can be fixed, that you can theoretically do HFT at the same speed as you are now without any quote stuffing if only you had perfect code.

Let's take a vaguely similar example of streaming audio. I worked on a near-realtime streaming audio product for a while so it's something I'm familiar with. Latency and reliability trade off pretty directly. The product I worked on defaulted to a latency of two seconds. This made it pretty robust against network hiccups like wifi interference, but the substantial lag made it annoying or just plain unsuitable for some applications. It was possible to adjust the latency (although this was an advanced technique to the extent that we hid the setting pretty thoroughly) and you could get it down to a fraction of a second and still have the system work overall, but it would be much more prone to dropouts due to network problems.

Now, there are some things that could be done to improve this problem absolutely, like adding forward error correction to the protocol, but nothing is going to eliminate the tradeoff entirely. Because of that, I'd never describe it as a bug. If someone complained about the latency, I told them that it was an unfortunate but necessary consequence of ensuring reliability. The latency can be eliminated easily, but at the cost of less reliability.

Likewise here, quote stuffing is an unfortunate but necessary consequence of HFTs. It could be lessened or eliminated by making HFTs less HF, but you don't want to do that, because you feel that the Hest possible F is more important. I call the writing weaselly because it portrays quote stuffing as completely unavoidable, but the real scenario is that quote stuffing is completely avoidable, it's just that you prefer to make more money instead of stopping it.


This is absurd. Every bug is the result of trade-offs. It's still a bug. The important point here is that this case a bug that typically costs quite a bit of money when it happens, and they try to avoid it. HFT devs should be better, usually are, fail sometimes - like any other dev. Duh.


You don't have to be that heavy handed about it. One simple check is to see if your new order is basically the same as the last order you just canceled (same price and side of the market). If it is, and it has been less than some delta, don't send the new order. I think this sort of duplicate detection is relatively common among HFTs but I could be wrong.


The low level of opacity when it comes to electronic trading makes it difficult to make any sense out of exactly what is (or perhaps isn't) happening, which I think is fundamentally the problem.


Do you mean the high level of opacity? There is considerably more opacity than in non-electronic trading, surely (everything on record, all data public, no effect from personal relationships etc)?


I can't tell whether you're suggesting that "HFT" (heavily automated electronic trading) is more opaque than human trading (ie, the specialist system), but for what it's worth there's a right answer to this question: human market makers are WAY less transparent than algorithms are.

Here's a benchmark: human market makers rigged the public markets so that tradable instruments were quoted in fractions of a dollar, rather than in pennies, so they could surreptitiously increase spreads. It's hard to get more brazen than that.


Human specialists and locals usually either had a monopoly on their product or were buddies with everyone in the pit so of course they would collude. Even on the NASDAQ screen they would collude and intimidate anyone showing tighter prices for showing "Chinese quotes": http://www.acsu.buffalo.edu/~keechung/TEM/Journal%20Articles...

Anonymous limit order books are far better. Nobody's price is better than anyone else's and machines have no problem competing spreads down to near-zero.


human market makers rigged the public markets so that tradable instruments were quoted in fractions of a dollar, rather than in pennies, so they could surreptitiously increase spreads.

I don't think this is right. Erh, obviously U.S. equities traded in quarters and eights (and later sixteenths before decimalization), but to say that they were using a weird fractional system because they wanted to rip people off isn't documented anywhere in the literature. They could have moved to decimals and still kept the minimum price variation above a penny†, or they could have just added more fractions (32nds, 64ths, 128ths) that would have moved the minimum price variation into roughly the range you get by quoting in decimals. If you think quoting in 128ths sounds silly I won't argue with you, but it's something they do actually do for the US treasury futures contracts on the CME††.

I suspect that they just used what worked. In a market of human traders, it's much easier to deal with large round numbers. I think that's why odd lots were treated specially too. E.g. It's a lot easier for humans to keep track of 5000 shares at 50 1/2 than in 100 small orders from 200 shares to 5 shares all priced between 50.40 and 50.60. As markets adopted technology, they naturally moved to a smaller minimum price variation (and eliminated the special treatment of odd lots) because computers don't have any problems dealing with an aggregate of small orders vs one big order.

† They are actually talking about doing this for some small cap stocks later this year. http://online.wsj.com/news/articles/SB1000142405270230427530...

†† ctrl-f "Quotation Practices" -- http://www.cmegroup.com/education/files/understanding-treasu...


https://www.sec.gov/litigation/investreport/nd21a-report.txt

Search "odd-eighth".

Decimalization drastically reduced spreads. Scott Patterson suggested in _Dark Pools_ that it literally put many human market makers out of business; in other words: those human market makers were kept afloat by the gross inefficiency of quoting prices in 8ths and 16ths, an inefficiency that was taken out of the hides of investors.


I was really just trying to make the points that 1) the minimum price variation of 1/8 of a dollar probably made sense at some point in the past (NYSE being started in the early 19th century) 2) sometimes it's more than just market makers who want larger tick sizes (consider this post from a managing partner at Andreesen Horowitz†) and 3) you can reduce the minimum price variation (the "tick size") without doing decimalization (i.e. smaller fractions) and vice versa -- you can set the minimum price variation at sizes larger than a penny even with decimalized quotations.

http://blog.pmarca.com/2013/03/26/unshackle-the-middle-class...


I'm not sure what this comment means. You said, earlier: "but to say that they were using a weird fractional system because they wanted to rip people off isn't documented anywhere in the literature." I think I refuted that, effectively.

Can you artificially inflate spreads without making inflated spreads the explicit law of the market by quoting in 1/8ths? Sure. Is that what happens? No.


I thought you were making a distinction between quoting in fractions of a dollar vs quoting in pennies. It isn't really material. If you quote in fractions but allow the minimum price variation to be 1/128th of a dollar you have a market that allows for tighter spreads than one in which the minimum price variation is a penny. There is nothing wrong with prices in fractions of a dollar. What matters is the minimum price variation a.k.a. the tick size.


It is material, because the fractions of dollars we're talking about were bigger than 1/100th of a dollar. Clearly, they could have moved to sub-decimal fractions, but the point isn't "fractions" versus "decimals"; it's "human market makers fixed the fractions at 1/8ths and 1/16ths of a dollar to artificially increase the spread", versus "algorithmic trading competed the spread down below a penny".

You implied that maybe humans had been using fractions because they were easier to deal with. No. They were using the fractions they were using in order to skim extra money off trades, and they fought tooth and nail to prevent competition from reducing spreads below 1/8ths --- in fact, if you read the link I showed you, it wasn't even 1/8ths; it was 1/4s!


What I meant to imply was that the emergence of fractional dollar tick sizes was probably due to the fact that they were easier to deal with before the advent of computerized execution, clearing, and settlement. By the mid nineties most of that had been automated and there was therefore no good reason for such large tick sizes. When humans were still doing all trade processing manually, it made sense to have a system that incentivized fewer large trades (vs more smaller trades). Large minimum price variations and round lot requirements would have been a means to that end.


Though it's worth noting that transparency wasn't the hallmark of earlier systems.


Except, once in a blue moon this happens when the mean signal approaches the threshold and stays there for a while:

I'm having some trouble with the contention that this is just such a rare thing. I do a lot of audio signal processing so I'm used to dealing with noisy signals at Khz rates. Threshold fluctiations are a fact of life if you're trying to do things like pitch detection, envelope following, or sidechain compression (all of which have obvious analogs in the trading context).

Obviously you don't want to impose too brutal of a low-pass filter on your input because you would miss important signals, but what prevents you from running the input signal to an LPF in parallel and keying a noise gate when the amplitude of the low-frequency (trend) signal drops? This is such a common problem in dealing with audio that we use specific devices called de-essers for recording vocals, because sibilant sounds otherwise tend to throw off other components of the signal chain: http://en.wikipedia.org/wiki/De-esser

Ironically, you mention using XKCD-type graphs because 'nobody reads' the explanations and caveats, but just looks at pictures. I wonder if you aren't missing a trick here by over-engineering your HFT systems for speedy response at the inevitable price of positional confusion. Is it possible that you're using Infinite Impulse Response filtering and running into denormalization problems as a result? If so, can you not rectify the problem by adding a small amount of noise?

I get your point about the negative aspects (for the HFT trader) of quote stuffing, how it can easily lead to being on the wrong side of a trade or punishment by the broker for flooding the output channel with unintended orders. But, you want to make money from automated signal analysis, that's your problem in much the same way that a sound engineer is responsible for preventing transient pops and feedback howls in an audio network. Sure, these problematic signal conditions are statistically rare, but when you're dealing with a high volume of samples you'r going to run into statistically rare conditions on a regular basis - and indeed you mention that this is the sort thing you might expect to happen once a day.


I think you are right on the money and most traders do use techniques analogous to the ones you describe when designing models:

http://www.decal.org/file/2945

"Alpha is often nothing more than taking commonly available data and mathematically encoding it in a signal correctly. Correct often means something as simple as using an rate of change instead of a difference, normalizing a value, smoothing a chaotic signal with an EMA, using a heteroscedastic weighted linear regression instead of a simple regression, or handling all numerical errors or edge cases."

It's not always easy to get this completely correct in every situation and traders face a reward function that doesn't necessarily reward correctness so much as avoiding Type II errors.


I feel like I'm in the wrong job - clearly I should be routing ticker prices into my mixing desk and playing the market like a cheap violin instead of being an underpaid sound engineer.


this is an interesting way to look at the problem but in my reasonably informed opinion, flawed.

it seems like this is working backwards from the assumption that quote stuffing must be benign activity, and then trying to rationalize it with signal/noise theories about finding the fair price level.

it is totally inaccurate to say that there is more load on the sender than the general public, and also misses the mark on what the DOS theory alleges.

the exchange bears considerably more load than the sender of orders. the exchange must (1) ack the order, or reject if any flags are populated out-of-spec, (2) add the order to the order book, (3) check for potential matches based on price of order, if so, cross and update feeds, (4) if non-marketable, publish out to (a) depth-of-book feed, (b) SIP feed, etc.

the DOS theory is that excessive activity in a single partition can slow down the publishing of data in that symbol range, which would mess up the quoted price for the exchange in a symbol, which would affect orders priced on the exchange, or anyone using the price in a derived manner (at a dark pool etc).


I'm not making the assumption that it's benign. It's based on my own experience - I've quote stuffed hundreds of times in simulation, and I'm sure it happened in production as well. It's a hard bug to completely avoid.

Interesting - the DOS attack is on the exchange rather than other market participants. But could you be more specific on how this helps you make money?


There used to be games where you would attempt to get one order connection on one exchange gateway and then another connection on a different exchange gateway. You'd spam with one connection to slow down all the other participants on that gateway, and use your other connection for your real trades.

Other games involved spamming open auctions, settlement trades etc.

This gaming was a very brief phenomenon (because it is stupid, doesn't make as much money as traditional trading, and trivial to foil). The exchanges got rid of it almost immediately but it still gets passed around as folklore for one of the "predatory" activities that HFT does.


Most exchanges are structured something like:

a.) Each trader has one or more gateway processes that do the order checking/rejecting.

b.) Each gateway maintains a queue of messages for the matching engine.

c.) The matching engine/order book for a product pulls one message at a time from each gateway that has a message waiting on its queue and processes it before spinning back to the first gateway again.

So a trader blasting the exchange with orders isn't going to impact other traders who also place orders around the same time. She'll only be slowing herself down.


DOSing the exchange will get you kicked out in no time.



Obviously, well written HFTs avoid this by introducing some hysteresis to the signal. i.e. Insert an order when the signal crosses 0.5, but then only delete it if the signal falls below 0.4.

Chris: You've also missed the most common cause of these issues in my experience - feedback from identifying your own orders in the public stream. HFTs need to filter their own orders out of the orderbook they are trading on, or a feedback loop can begin. This can be difficult, since some exchanges don't provide a clear way to identify your own order, or it may be the case that you receive the public UDP packet before the private TCP packet that contains the ID required to match the two.

For example (a basic one), say I have an algo that calculates the midpoint based on the best bid and ask volumes. And can only insert within x distance of the midpoint. I add a bid to the top of the book for a 1 lot, get the UDP public depth update, don't know whether this is my order or someone elses. So I re-calculate the midpoint, which pushes the midpoint upwards (greater bid volume), which means I then pull my bid, which means I recalculate the mid again, then re-insert, etc. etc....


I actually mentioned this briefly - see the discussion of the "reconciliation step" in the "Quote Stuffing is not a DOS attack" section.

I guess I assumed everyone's reconciliation system was as good as the one I worked on. But you make a very good point, as well as a meta point - this stuff isn't easy.


Sorry, missed that bit


Sure but what if someone else is doing something in the market that perfectly lines up with the edge conditions for your place and cancel thresholds? Nothing is foolproof. If you are operating at a large enough scale this will end up happening a few times a day, especially when you have multiple agents interacting in ways that can't be predicted. A feedback loop can still happen between two unrelated traders running similar rules like "if I'm not the best offer, improve it by a tick".


The 2. in the series, if I understand it right, is that they overwhelming the derelict SIP feed between market locations, increasing the benefits that their own high speed connections brings. If true, that does make money because they can act "in advance" at other markets before the artificially slowed SIP data gets there.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: