

LSE Halts - ashishb4u
http://www.londonstockexchange.com/exchange/news/sharecast/news-detail.html?newsId=4072679
http://online.wsj.com/article/SB10001424052748704150604576165753761876730.html?mod=googlenews_wsj
======
ig1
The problem is that the LSE can't hire top tier talent. There are plenty of
people who are experts in building hugely concurrent and fail-safe exchange
software, and they pretty much all work for investment banks.

The LSE can't afford to compete with $1000/day salaries so they end up hiring
second-tier developers. And unfortunately second-tier developers are a lot lot
worse than first-tier developers.

~~~
laktek
I don't think it's about talent, but more of a problem of proper planning and
execution. It seems things had been rushed to meet the tight demands, causing
the chaos.

When LSE acquired MilleniumIT, to redo the platform, they claimed to have
developed world fastest trading platform[1] and had enough talent under their
belt developing trading systems around the globe for last decade.

[1]
[http://www.wallstreetandtech.com/exchanges/showArticle.jhtml...](http://www.wallstreetandtech.com/exchanges/showArticle.jhtml?articleID=220000655)

~~~
ig1
None of the exchanges have the required talent. Being the best exchange
technology company is like being the best team in a minor league. If you're
good at extreme low latency stuff you're going to be working for a bank.

At a bank being great at low-latency can mean tens of millions of dollars of
profit for your trading desk and a huge bonus for your personally.

At an exchange being great a low-latency means ... well not much, your
exchange gets a bit more flow and you get a pat on the back.

~~~
chollida1
> At an exchange being great a low-latency means ... well not much, your
> exchange gets a bit more flow and you get a pat on the back.

Exchanges pay out bigger bonus' than you'd imagine. 7 figures for a programmer
is high but no where out of the ordinary. Says a guy who has worked for an
exchange.

~~~
ig1
I seriously doubt any exchange is regularly paying developers multi-million
dollar bonuses.

Which exchange did you work for, I don't see one on your linkedin profile?

~~~
_debug_
Exchanges often use (relatively) esoteric hardware + real-time OSes, such as
the Tandem NonStop series. Experience with such a platform + domain knowledge
about the finance industry is a rare combination, so I find it plausible that
a couple of core expert programmers on the stock exchange's system get
compensated quite well. High six figures + 80% bonus == seven figure, I guess?

------
famoreira
Does anyone know if this halt is related to the recent switch over to the new
Linux based platform? (<http://news.ycombinator.com/item?id=686116>)

~~~
drstrangevibes
it is a new platform, hard to tell but I wouldnt go so far as to say linux is
to blame, its more likely to be exchange software, they've gone from native to
fix 5, so its using a new protocol, new os, new software stack and most likely
new hardware stack

~~~
tallanvor
I don't think famoreira was trying to blame Linux. --That's just the part of
the new system that you can reference easily.

~~~
famoreira
Yes, I didn't mean to blame Linux. I was just referring to the system in the
way it was made famous (replacing a Microsoft based system with a Linux one).

------
mtkd
"And as a final note, solely for the seriously nerdy techies, there's been
some talk about the failure being due to F# error."

<http://ftalphaville.ft.com/marketslive/>

~~~
alecco
I still don't understand how they choose languages with garbage collection for
systems that should be real time or similar.

~~~
davidw
Erlang is not "real time" in the hard, embedded systems sense, but it does
pretty well and has GC.

Here's a bit more information:
<http://www.erlang.org/faq/academic.html#id55755>

I don't know what sorts of 'real time' properties trading systems are required
to have - can anyone talk about it with a bit more in-depth knowledge?

~~~
IDisposableHero
I don't know much, but for high-frequency trading, fast is never fast enough.
response times are measured in microseconds.

[http://richg74.wordpress.com/2010/10/25/speedy-new-lse-
tradi...](http://richg74.wordpress.com/2010/10/25/speedy-new-lse-trading-
system/)

[http://www.h-online.com/open/news/item/Linux-boosts-Stock-
Ex...](http://www.h-online.com/open/news/item/Linux-boosts-Stock-Exchange-
trading-speeds-1126476.html)

~~~
alecco
Speed isn't all. The system should be based in algorithms with manageable
worst case time.

------
timrobinson
"Continuous trading has now resumed in all order driven trading services."

------
alecco
Sri Lanka? [http://uk.reuters.com/article/2009/09/16/uk-lse-it-sb-
idUKTR...](http://uk.reuters.com/article/2009/09/16/uk-lse-it-sb-
idUKTRE58F4TC20090916) (via /B @ftalphaville comments)

------
ashishb4u
Nothing official yet.
([http://www.ft.com/cms/s/0/9bf8033a-40bb-11e0-9a37-00144feabd...](http://www.ft.com/cms/s/0/9bf8033a-40bb-11e0-9a37-00144feabdc0.html?ftcamp=rss#axzz1ExjV27Jt))

------
alecco
What if all these systems were open sourced? Including all the load tests.

~~~
rbanffy
> What if all these systems were open sourced?

LSE considers its software to be a competitive advantage.

~~~
drstrangevibes
.....er sure

~~~
rbanffy
When they make it work, it could be ;-)

------
tpatke
Where is the fail over? If ever a system needed 99.9999...

~~~
JonnieCache
Failover in this scenario would require instant, 100% replication of a near-
zero latency system.

I'm no expert, but I'm fairly sure having trades 'disappear' or any other loss
of consistency would be worse than downtime in this case.

------
drstrangevibes
this is the official place for updates, although its not very responsive at
the moment,probably due to load
[http://liveservice.londonstockexchangegroup.com/en/incidents...](http://liveservice.londonstockexchangegroup.com/en/incidents/active/lse)

~~~
pnathan
It's been archived - here's the link to this incident in the archive:

[http://liveservice.londonstockexchangegroup.com/en/incidents...](http://liveservice.londonstockexchangegroup.com/en/incidents/archived/85-friday-25022011-075443)

------
JonnieCache
I wonder if it's HFT related? A group of rogue algorithms gone exponential?
There are certainly people worried about that...

