

Advice on the implementation of a real-time trading system [closed] - wslh
http://stackoverflow.com/questions/14254524/advice-on-the-implementation-of-a-real-time-trading-system

======
boothead
It's not the orders you need to monitor from the sound of it (at least once
the trade is complete), it's the position that you hold in the market vs the
current price.

* Actually the previous statement could be wrong: You might also want to do this on a per order basis, although they are on some sense functions of each other.

I would suggest the following:

For any market which you have a position in, subscribe to that market's price
feed. I'd use zeromq for this, but there are plently of pub/sub libraries and
products. The previous point implies that you are republishing your market
data feed into specific channels. How they are mapped depends on many factors,
but basically you want to de-multiplex the market data so that you are able to
pick out one market's price data as a stream of updates from the whole.

Once you have a feed of the market with a position you can use it to transform
the stream of market data to multiple streams of values. You might have new
steam of PnL values for each order you have on the market, or a stream of
price differences or whatever, you then re-publish those streams. Depending on
how many of these derivative streams you have and the rate of data, you might
want to do some downsampling too.

What I've described above is pretty similar to the market data architecture in
hedge funds that I've worked at. Have a look at the ideas behind function
reactive programming
<http://en.wikipedia.org/wiki/Functional_reactive_programming> for more info
on stream processing.

For the front end, you'll bridge the particular stream of data to a WebSocket
(probably translating it to json or a binary data format that javascript can
understand at the same time). I don't know how you do that in RoR, I'd use
gevent or eventlet in the python world. You'll then need to route this data in
some way to a representation of the order/position in the front end. I'd
suggest looking at knockout.js, ember.js, angular or something similar as
those declarative style libraries will massively reduce the complexity of your
client side coding.

Last bit is visualization and there's really only one sane choice for this
imho: D3 and libraries built on top of it. Look out for
<http://square.github.com/cubism/> and <http://square.github.com/crossfilter/>
and <http://code.shutterstock.com/rickshaw/> for those bits.

Bit of a whirlwind here, hope it helps!

~~~
brutuscat
Regarding the front end, it's exactly what I'm doing right now. I receive the
data, do some processing, pack it into json format and wire that using
WebSockets to the client. It works really well.

------
eduardordm
He needs to get in touch with the stock exchange (and brokers) before jumping
into any implementation conclusions.

He is not a broker, you can't really interact with any exchange directly. He
needs to use some bank (which will already do 99% of what he is implementing).

Without knowing exactly how the market systems work, this is a toy, he should
have talked to someone BEFORE writing any code. Things are way more complex
and simpler than it looks. It's extremely simple to use bank services to do
that for you and close to impossible for a developer to just jumpt into
writing trading software without a bank backing it.

~~~
RaphiePS
Someone ought to create a "stripe for stock trades" to make building stock
bots easy-peasy for any developer looking for a challenge. I'd certainly use
it.

~~~
eduardordm
Sounds like an idea PG would fund. The problem here is not even money, is
having enough influence to get people and banks to trust you - and get a deal
sweet enough to be profitable.

~~~
brutuscat
And that's exactly why is not that easy to get in touch with banks and brokers
too.

"Financials" sucks and they hide every inner workings...

------
dunster
You might be interested in zipline, an open source backtester written in
Python. Zipline, Quantopian's open source backtester (yes, I work at
Quantopian).

The backtester is designed to also be a trading engine. You feed zipline data
in "events" where each event is a point in time. Zipline doesn't care if the
events are live or replayed from a database - it processes the data and
executes any trades in the order book.

The tricky part here is that zipline is more aimed at implementing trading
algorithms. You'd have to write a simple algorithm to implement the buy/sell
orders coming in.

<https://github.com/quantopian/zipline>

------
pknerd
Some of my similar architecture questions were closed by Mods. I don't know
where one can ask such questions?

~~~
codegeek
"I don't know where one can ask such questions?"

May be start a SO type site which focusses specifically on Artchitectural how-
tos ? Like "How to design a real time trading system", "how to design a
supply/chain software" etc. What do you think ?

~~~
tkahnoski
I think the How To format might be too limiting as their are many possible
answers. A wiki format with embedded diagramming targeted at architecture
would probably suffice once you arrive at some standard for page structure.

This small toolset could also elaborate various business models and business
processes as well.

------
brutuscat
Interesting feedback I already got:

[https://groups.google.com/d/topic/celluloid-
ruby/9RHPmYkxGtU...](https://groups.google.com/d/topic/celluloid-
ruby/9RHPmYkxGtU/discussion)

[https://groups.google.com/d/topic/rubysur/ZJoZxDVcRYU/discus...](https://groups.google.com/d/topic/rubysur/ZJoZxDVcRYU/discussion)

~~~
aidos
Lmax was the first thing I thought of too but I don't really see how it
applies here.

Celluloid looks interesting have you played with it yet?

------
RossM
While it's certainly an interesting question, I presume you're posting it here
to garner support for it being reopened (as far as the rules go, I agree with
the closure, it's very open-ended). It might fit better on
<http://programmers.stackexchange.com>.

I know little about stock markets but I quite like the idea of spawning
workers to monitor orders after being placed. These could then update a page
live to the customer.

~~~
aidos
I too think this is an interesting question and I'd love to hear how people
would approach it.

How long do the orders need to be monitored for? Indefinitely? If so, is it
really feasible to keep them all running? How much data do they need to
analyse / aggregate to make their decisions?

Very interesting question.

~~~
brutuscat
For instance orders would be open from week into week if the user enables it
(each week).

In my case "decision making" is only based on some fixed price threshold in
order to provide some risk management. So the amount of data to aggregate
would be minimal. But the data to analyse came directly (as a huge stream)
from the markets.

------
OldSchool
From some of the comments I see, let's not read too much into this. He's not
asking how to build an -automated- trading system, just one that's realtime
and all analysis and decisions are made by the user. The best I've seen that's
commonly available is "Trader Workstation" and it's free with your account at
Interactive Brokers. It's written in Java.

------
yesbabyyes
Yesterday I learned about Postgres' LISTEN/NOTIFY functionality. I'm not sure
about your architecture, or how many listeners/watchers Posgres supports, but
it might be that you can just listen to changes in your database -- that might
be the most lightweight solution.

Edit: Here is a small tutorial: [http://bjorngylling.com/2011-04-13/postgres-
listen-notify-wi...](http://bjorngylling.com/2011-04-13/postgres-listen-
notify-with-node-js.html)

~~~
meaty
If you're trading, keep it all out of the database. It won't scale and latency
is terrible and unpredictable regardless of the engine.

Keep the database for storing stuff that you don't need in ram. Keep the rest
in ram and message queuing/pubsub bus architecture.

Plus events in a database engine are a pain in the butt to write tests
against.

------
rhapsodyv
I have made one in ruby for my trades. The key point of the architecture was
to do all asynchronous with queues and workers. It worked for me because the
network time was big and I don't do high frequency trading. I have used
rabbitmq and ruby with event machine.

------
pencilcode
You could use one of the realtime api's, like pusher or realtime.co. They can
handle the scale. Otherwise erlang/otp looks like a good fit

------
dschiptsov
Use Java, of course.

~~~
brutuscat
Why "of course"?

~~~
meaty
Off the top of my head, the op is probably talking about the off the shelf
stuff available for mq stuff and integration which is unsurpassed in java,
plus the very low latency IO.

TBH when I write java, I usually write virtually no code: I just glue other
shit together.

I don't get why people have so much against Java - it gets shit done in
incredibly short amounts of time.

~~~
chewxy
Until you have to come up with your own algorithm. Then you're stuck with
ohgods-so-many-.java-files

~~~
meaty
Depends. I wrote a rather complicated corridor based routing algorithm for a
courier company in the UK which worked on real time traffic data. It was only
11 files and 12,000 loc and that was their entire business model.

------
BUGHUNTER
Wouldn´t it be much more interesting to spend that precious brainpower into
something that actually solves real problems like war, disease, crime,
militarism, xenophopia, lack of education, etc. instead of building another
greed-engine that will trigger even more poverty, exploitation and repression?

Of course everybody is free to act on his own karma (in a few countries) - but
you will have a much better life by doing some more useful work that will
benefit all human beeings, not only a few criminals that build weapons of mass
murder just for the mentally poor reason of making more and more money.

Programmers haz powers to push human evolution, don´t waste your life on
golden lies, that will turn into deadly dust in the end!

~~~
davidmr
This kind of sanctimony is tiring. Every time one of these threads comes up
here, it inevitably devolves to a point where someone brings up something like
the above. There are those of us who actually enjoy these kinds of problems in
and of themselves. I don't consider what I do particularly noble, but I'm paid
to solve very interesting and unusual problems in a very fast-paced
environment, and I don't feel the need to apologize for it.

I'm under no illusions that I'm doing God's Work, but if the bar for useful
work is something that will end "war, disease, crime, militarism, xenophopia,
lack of education", I think >99% of us fall quite short of it. Maybe I fall
shorter than you do (you don't mention what it is that you do), but I still
enjoy my job.

