
Bloomberg Runs on 25M lines of Fortran (2006) - charlieirish
https://etrading.wordpress.com/2006/06/01/25-million-lines-of-fortran/
======
apaprocki
Sigh. Bloomberg-er here -- this article is old. There is certainly a lot of
Fortran remaining in sections (you don't rewrite working code with C linkage
in another language just because -- you rewrite it if you have something to
gain), but it is nowhere near as bad as this old article makes it sound. Many
teams are C++ only and have been for a long time (see our open-source code on
GH for a sample). Keep in mind that if you deal in absolute numbers you are
missing the bigger picture -- the actual real number will be much smaller and
it will be an even smaller _percentage_ of the total codebase.

edit: Can mods put (2006) in the title? Seems reasonable...

~~~
forgetsusername
> _Sigh. Bloomberg-er here -- this article is old._

Why be offended, rather than excited to get to come in here and to expound on
what you do for a living? It's what makes the comment section great.

~~~
nkassis
I think this kind of obsolete information may also hinder them when they try
to hire. Good engineer may have a bad perception about how it is to work there
and overlook them.

They could change that perception with being more open about how they go about
developing software (like their GitHub account). But these article showing up
every one and a while goes against their efforts hence why I can see why OP
may get annoyed about it.

~~~
oneJob
Last time I was in the Bloomberg building much of their Dev Dept was
underground, and they have a neon art display which is supposed to simulate a
sunset, or natural light, or something.

Sure, visiting Bloomberg and getting the tour leaves you with a wonderful
expression, but those that I know that have worked there in the past wouldn't
compare the work environment to Google.

Bloomberg is much more of an "East Coast" company, and feels like it.

Having said that, I had a Bloomberg Terminal subscription for five years. I
friggin' loved it. Regardless of whatever might be behind the screen, it's
still an amazing, powerful, gorgeous tool. I miss my B-Unit, oh so badly.

~~~
apaprocki
Not sure what you're talking about. The "underground" lower level is all
conference / presentation rooms. No one works there. The simulated sunlight
you're referring to is a large art piece installation in the building and the
creator chose to simulate light as he saw it during a particular sunset (in
Arizona IIRC? It's been a while...)

Check out the new SF office we opened up if you're looking for a more West
coast feel.

~~~
oneJob
I stand corrected. It was really too long ago (+4 years), I should really have
double checked my thoughts before posting. Gracias!

------
radmuzom
The article seems to assume that there is something "wrong" with running on
25M lines oF Fortran. It does not provide any technical reason why Fortran is
not the right tool for the job. Why does this need to be "Web 2.0"?

~~~
DrJokepu
I client of mine still has a fairly large piece Delphi of code in production
and I must say it's a bit of a liability for them. An example of a technical
problem they face is getting the particular version of Delphi they use work
with TLS (and that's becoming a problem now as everyone's starting to disable
SSLv2 & SSLv3). The fact is that they simply wouldn't face this kind of
problems if they have ported their code to something like Python, C# or Java.

On the non-technical side of things, they also struggle with hiring developers
who are capable of (or willing to) work with Delphi, especially junior
developers. A significant portion of their staff also doesn't know how to work
with Delphi, so reassigning resources to match the needs of the business is
often a challenge.

I imagine these issues would probably apply to Fortran as well.

~~~
RogerL
Fortran is entirely modern. Last standard is what, 2008? And any compiler will
just eat any previous version without complaint. You can take a huge swath of
'77 code, put some 2003 and 2008 in there, and it will all just compile.

Python? Good luck. Every frigging month (practically) it's a breaking change.
You have to work furiously just to keep your infrastructure working, or
purposefully draw a line in the sand (2.7!).

I run Fortran every day (via NumPy and other packages), and some of us are
investigating just switching to it for some projects (we need the math speed,
and all the C++ matrix libraries involve compromises of one form or another).
There's no reason I and my peers wont be doing the same in 50 years. It works,
its fast, its debugged, and it is easy to use things like Python 5000 (by then
over 70% of people will have made the transition from 2.7 to the 3.x
version!!!) to do all the glue work that Fortran is not optimized to do.

Fortran codes ain't ever going to be a liability[1]. Python 2.7 on the other
hand...

[1] because of being Fortran. Poor coding style was endemic in Fortran77, and
that can be crippling, but to a large extent that is language agnostic. I
acknowledge the required short variable names of 77 doesn't help, but people
would not comment what the variables mean, and that is the real problem (IMO).
Poor structure is another problem (masses of gotos). Yes, 77 doesn't make your
life easy there, but discipline is possible. We can quibble about all this; my
point is the 'modern' languages are going to be old quite soon, and we will
face _real_ porting problems then. Move 20,000,000 lines of JS and Python?
Good luck.... Get a Python 2.4 code base working with 3.9? Ah, hmm, well .....

But Fortran? We build it and run it every day, easy peasy.

~~~
icebraining
_Python? Good luck. Every frigging month (practically) it 's a breaking
change. You have to work furiously just to keep your infrastructure working,
or purposefully draw a line in the sand (2.7!)._

What? There was exactly one breaking point, from Python 2 to Python 3. Which
"monthly breaking changes" are you talking about?

~~~
kragen
Posting this censored comment for the good of the community; it seems entirely
reasonable, although a bit overwrought.

"Gurer jnf rknpgyl bar oernxvat cbvag, sebz Clguba 2 gb Clguba 3"

Bar oernxvat cbvag?

Gur znwbevgl bs gur yvoenevrf gung V hfr qba'g rira jbex cebcreyl va Clguba 3.
Gur fgnaqneq Zlfdy yvoenel sbe vafgnapr, juvpu vf hfrq va gur znwbevgl bs BEZf
(vapyhqvat FDYnypurzl) vfa'g vapyhqrq va Clguba 3, juvpu zrnaf lbh arrq gb
znxr znwbe punatrf/gel gb hfr n qvssrerag yvoenel.

Gur pnfr bs znal fgnaqneq yvoenevrf jrer punatrq, oernxvat ybgf bs pbqr.
Rknzcyr: PbasvtCnefre gb pbasvtcnefre.

Rira fvzcyr guvatf yvxr cevag 'grfg' vf abj cevag('grfg')

Gurfr ner bayl n srj rknzcyrf. Gurer ner ybgf bs bgure znwbe punatrf gung
oernx 2.K pbqr.

V unir orra hfvat Clguba 2.K sbe nyy bs zl cebwrpgf naq jnagrq fb onqyl gb
punatr gb 3.k sbe zl arj cebwrpgf. Ubjrire, orpnhfr bs nyy vapbzcngvovyvgvrf,
V pna'g.

V qba'g guvax V'z nybar ba guvf rvgure. Clguba 3.K fgvyy vfa'g orvat hfrq ol
gur znwbevgl bs qrirybcref.

~~~
amyjess
None of that contradicts the assertion that Python's only breaking point is
Python 2 -> 3.

Is there some reason you ROT13'd it?

~~~
kragen
No, none of it does.

I rot13ed it in case there's a dead-comment-reposting-killing algorithm on the
loose.

------
ig1
(In a previous life I worked at Bloomberg)

I don't remember the exact figures but 25m lines of Fortran seems plausible,
but it was still the minority of the code base. The majority at the time was
in C and C++, with probably a few million lines of Perl and Javascript as
well.

Plenty of the Fortran dates from the 80s, rewriting it in a modern language
would be a huge project which lots of risks and limited upside. No significant
new fortran has been written for a long long time (>10 years).

It's also a mistake to think of Bloomberg as a single piece of software, it's
closer to an app store, there's are thousands of specialist apps (functions in
Bloomberg terminology) and most users only use half-a-dozen, but which half-a-
dozen varies significantly between users.

This makes it very hard to displace wholesale. There are individual pieces a
competitor can go after but identifying a subset of functionality that's
enough to convince users to switch is hard.

There's also a lot of stickiness which goes beyond pure functionality. Network
effect and brand are also key parts (having a Bloomberg Terminal is a status
symbol).

~~~
apaprocki
Hi Imran :) The number is actually much smaller, but it is still a minority.
We have many times more lines of server-side JS than we do Fortran.

~~~
ig1
Hi :)

From what I recall there's a ticker somewhere which tracks lines of Fortran
code which I vaguely recall being in the millions.

------
bradleyankrom
It's a lot of C++ today. In fact, they have open-sourced a lot of the in-house
tools they use, eg bde:
[https://github.com/bloomberg/bde](https://github.com/bloomberg/bde)

~~~
nly
I'm kind of surprised, but happy, to see they're using WAF as a build system.
I've really enjoyed using it in the past.

------
PaulHoule
I have been looking at some projects at Bloomberg from the outside and even
those the projects are interesting I don't think I'd want to work their
because of bureaucracy and inertia -- it's the kind of place where you'd need
to get approval to install a text editor on your machine.

I have found that often "high profits" get in the way of customer service
because they are a disincentive to "quality is free" thinking and make it
possible to sustain the unsustainable for way too long.

To take an example, Cable companies are so profitable that they think nothing
of the cost of replacing cable boxes that break, or of excessive truck rolls.
These not only cost money but they anger consumers. My mother-in-law quit
cable in disgust and switched to OTA TV after she had three cable boxes burn
out in three months and would have to go stand in an long line to return it
and pick up a new one.

Quality is free and screwing up is expensive -- even if you can afford to
screw up it costs you customers and it costs you employees. But again,
turnover is no problem if you are making enough money you can afford to spend
5x what things should really cost.

~~~
GCA10
Well said. Thanks for the perspective.

------
melling
This article is from 2006. I imagine something has changed in 9 years.

~~~
retube
You clearly haven't worked in a large enterprise!

~~~
melling
Sure I have. That’s why I said "something" instead of "a lot". They were
investigating C++ in 2006 and I've actually talked to a C++ developer from
Bloomberg so something has changed. Just search their job listing:

[http://jobs.bloomberg.com/go/All-Open-
Positions/515100/?q=&t...](http://jobs.bloomberg.com/go/All-Open-
Positions/515100/?q=&title=developer&sortColumn=posteddate&sortDirection=desc)

~~~
baldfat
I still feel Fortran gets a bad rap. Fortran has been a work horse and is
extremly fast. (Mind you I learned Assembly on a C64 so Fortran looked so much
"nicer" back in the 80s and I don't program in it at all but it seems like
everyone treats it like it is some mothballed slow poke.

------
drglitch
BBG's competitive advantage is not data - in fact, it's pretty mediocre. Any
reasonably large player sources their own data feeds due to too many holes in
the former.

BBG's killer features are support and _chat_ - chat with other _trusted_
counterparties who also paid the admission price.

~~~
xntrk
And if you start talking about non-equity markets there is a lot of trading
done on BBG as well.

------
beezle
It would have been interesting to know if they are using Fortran 95 or the
more recent (and very modern) 2003. People who like to beat on Fortran rarely
know that it has evolved quite a bit since '66 and '77

------
matthewaveryusa
I worked at bberg for 4 years. Changed about 2 lines of fortran code. Some
people do more, others so less, but the norm is you'll spend a sliver of your
time in fortran.

Great place to work, too bad it's in NYC

~~~
infinotize
There's actually a SoMa office now (as well as London, which has been around).

------
baldfat
A little Wikipedia information:

Market share has grown since this was written in 2006.

Market share in 2007 26% and 2011 30%

Money: 315,000 Bloomberg Terminal subscribers (2 year subscriptions) worldwide
at $20,000 per user = 6.3 BILLION dollars a year!

EDITED for spelling

------
edu
The article is from 2006, should be noted on the title. And, anybody knows if
the situation is still the same?

------
stevoo
I have to disagree with having a company actually coming in and taking out
everything that bloomberg has been building the past years. This is an
extremely hard market to get it. Bloomberg is extremely dedicated at what they
do and they do not take competition lightly. They do use there massive power
to make the life of competition as hard as they can.

As for Bloomberg there terminal and all of the services that they provide is
top notch. I have a lot of friction with all of there data, and this is by far
the best i have worked with.

------
smanzer
Unhappy academic here - codebases with millions of lines of Fortran are very
prevalent here as well. It is both better and worse than a lot of people
think; like a lot of the other comments said, that code is rarely touched, but
when you need to touching it can be very painful. Modifying old Fortran is
rough, mostly because of that awful, awful IMPLICIT keyword. Though I will say
that the native multidimensional array support does make certain sections much
easier to read than corresponding vanilla C code.

------
pma
Bloomberg terminal and Thomson Reuters are certainly ripe for strong
competitors.They are basically operated as monopoly for decades.Yc,developers
and others should take on this challenge.
[http://www.nytimes.com/2015/09/10/business/dealbook/the-
bloo...](http://www.nytimes.com/2015/09/10/business/dealbook/the-bloomberg-
terminal-a-wall-street-fixture-faces-upstarts.html?ref=business)

~~~
KaiserPro
Traders value consistency and dependability.

Its not like you can throw together some trading system with an eventually
consistent datastore in the back.

the entire finance industry is based on trust. Part of that is trust that
bloomberg is correct.

Loosing a couple of trades without an audit trail crashes markets.

~~~
fennecfoxen
> Its not like you can throw together some trading system with an eventually
> consistent datastore in the back. Loosing a couple of trades without an
> audit trail crashes markets.

That's not consistency, though, that's _durability_. Consistency in banking is
basically the part where you don't allow people to spend money twice. That can
be "eventual" and violating it can be OK under some circumstances, like when
your bank issues you an overdraft (and applies a fee) or when your airline
sells too many seats.

So the eventually consistent data store in the back is fine... as long as it's
durable and as long as you're willing to invest man-years of effort in
understanding the implications of your selected consistency model so that you
don't screw it up and can perform operations which are compliant with some set
of business rules that the business understands and approves of, naturally.

~~~
KaiserPro
I'm talking about trades specifically.

A trading platform must have a 1:1 mapping of buyer to assets when trades are
agreed. You cannot have a situation where two people buy the same share, as it
requires unwinding. A trade must be atomic.

traders want to see all the bids/asks of the whole market. Otherwise you'll
effectively have a random assortment of tiny markets under one supposedly
united trading platform.

Thats the other thing, trades need to be reversible. There is a good
correlation between massive trade volumes and the need to reverse transaction.
Having an eventually consistent model is just not going to work.

Being overdrawn is not eventual consistency. The act of an overdraft is not a
failure of a bank to properly account for your balance, its a deliberate
business move to generate cash.

So no, its consistency, one price, one bid, one ask.

~~~
fennecfoxen
The concept of overdrafting is a combination of modern business rule and
ancient technological limitation (dating back to before-computers when
accounting was manual and checks and checkbooks were also big deal, which his
hella-eventually consistent). It can still serve as a useful metaphor,
regardless.

But if you care about operating the EXCHANGE ITSELF then your allusion to the
audit trail is a bit of a red herring, isn't it? In fact, the data store
itself becomes pretty irrelevant for the live trade-matching - presumably you
shard just the hell out of your data, keep the working set in RAM, keep most
of your stuff colocated in the same place and just solve for availability by
doing something crazy and hardware-intensive like having hot spares for
everything.

------
monopolemagnet
By comparison, this is 33% more LoC than a BWR/PWR nuclear reactor (Monte
Carlo method / quadruple integral (⨌) equivalent) simulator product formerly
known as CORETRAN-01.

~~~
monopolemagnet
Finite element analysis modeling is _really hard_ , even for nuclear engineers
whom can code.

A lot of scientific control system apps get built with LabVIEW or Agilent VEE.
If it makes beyond the niche consultancy stage, such things are often
(re)written in whatever an engineer happens to know: C, Python, Haskell, LISP,
[favorite language religion here].

Edit: BTW, just came across a supposed successor to VMEbus [0] (a standard
industrial data bus): VXIbus [1]

0:
[https://en.wikipedia.org/wiki/VMEbus#/media/File:VMEbus.jpg](https://en.wikipedia.org/wiki/VMEbus#/media/File:VMEbus.jpg)

1:
[http://www.vxi.org/specifications.html](http://www.vxi.org/specifications.html)

------
zimbatm
When the compiler code size is a fraction of the whole it would actually make
sense to improve on it instead of rewriting everything. Like Facebook did for
PHP and their HHVM. Obviously Fortran doesn't have a speed problem but it
could be improved in other ways like static analysis and nicer syntax
notations.

~~~
protomyth
Converting to later versions of the Fortran standard would get you a long ways
there. Plus with all the focus in 2003 & 2008 with C interoperability, you can
plug in later open source libraries.

[edit: I wonder what a modern version of RATFOR would look like]

------
shitgoose
In Fortran one can add two matrices: A=B+C. Try it in C++ (NumPy is better
that way, but still this is an extension to a rocky foundation). The amount of
freely available high quality math/statistical libraries for Fortran is
unmatched. Why in the world would you give up on this just because you can use
Google account? Google Payments for settlement?? WTF? Settlement is a bit more
then credit card charge or moving money between checking accounts. Google
Fiance as a ticker plant?? Yea, right. Even Yahoo provides options data,
Google has just basic stock prices. Compare it with BBG that has _everything_.
Build terminal in Web Kit? Why???

Every time I see suggestions like this, I remember minions, rushing from one
framework to another - "koonga la mala makuna, koonga!".

------
jamesrom
And there are 6 million rivets in the Sydney Harbour Bridge.

------
josephmx
This reads a bit like the old "Google will rule the world!" blog posts,
doesn't it?

~~~
geekus_maximus
That's because it is old

------
tempodox
I don't know how comparable this is, but JaneStreet seem to be quite happy
with OCaml. I for one would give that some consideration before actually going
C++. And that's ignoring the question why they would depart from Fortran in
the first place.

------
USNetizen
People would be really surprised how much of the critical infrastructure of
this country in government and enterprise runs on technologies around half a
century old. This is nothing new. The IRS still uses millions of lines of
COBOL to process our tax returns, the VA uses millions of lines of MUMPS (M)
to store and process health and benefits records, and the list goes on and on.
It's not going anywhere soon, either.

People skilled in these legacy languages and technology stacks are also
amongst the highest paid and most in-demand in the country, at least here on
the east coast in places like New York and DC.

------
eb0la
I though Fortran programmers are more expensive than C++ ones; but
itjobsearch.co.uk says they aren't.

Fortan programmer: 45,000 GBP (3 month avg), +23% salary change from last year

C++ programmers: between 47K GBP to 100K (for c++ quants).With 3-5% average
salary change from year to year.

I don't know Fortran; but looks like the fame about its high costs comes from
mainframe hardware (which also runs C++, btw).

------
crb002
Bloomberg can write a one page Python flask/bottle wrapper around any of that
Fortran and serve it up as a microservice. They got Pang Ko from Mathworks who
is one of the top parallel computing data structure minds in the world. I'm
betting Bloomberg will be around until another player like JP disrupts the
market with a truely distributed trading platform.

------
oneJob
Just read the article. Yeah, a Bloomberg terminal does a lot (a lot alot) more
than scroll "news tickers, checking Bloomberg email, and trading." When the
author suggests that Google might make a play for market share (or some other
Web 2.0 player), it is clear that there is an enormous lack of understanding
as to exactly what Bloomberg is, as a product.

------
anonu
I didn't notice the "2006" in the title until I read that the terminal cost a
grand a month! I thought something was off!! Bloomberg Terminal costs more
like $2k/month per user nowadays... maybe more??

------
1971genocide
Wait.

How can you trivially conclude that its would be easy to provide the same
speed as Blomberg terminal ?

If I am not mistaken even shaving off fractions in terms of time has a lot of
value.

And maybe you need 25 million lines of FORTRAN code to achieve that efficiency
?

~~~
angdis
Yeah, they brush it off as a minor issue, and just conclude Google can handle
it!

This sentence makes me dismiss the entire article:

"The trading model and real time issues are just technical problems. Google is
smart enough to solve them."

~~~
spacecowboy_lon
Quite Google has never been about exactness if 1 in 10k searches serves a bad
result - they dont care.

And I have written mission critical Fortran software for big telcos' and tust
me nothing contreates the mind when one on Vint's reports nudges you and says
this had better be right or we are both looking for anew job.

------
meerita
I know Bloomberg is the Kraken of the softwares out there, but i wonder how
many lines lines would be in other langs?

------
spacecowboy_lon
Only 1500 coders I recall at British Telecom hearing oh we need to put extra
800 developers on a single product.

------
allenwlee
In my mind, Markit is the biggest future threat to Bloomberg

------
vasche
How is this news?

~~~
DanBC
Hacker news isn't news. You might want to re-read the guidelines.

