
Ask HN: Why isn't all stock trading done by algorithm? - Peradine
I can&#x27;t understand how a human, even with heaps of intelligence and education, is better at predicting which stocks will go up&#x2F;down than an algorithm that can look at all the data in the entire market. What skill do stock traders use that can&#x27;t be replaced by computation?
======
greenpizza13
Computers trade most effectively with technical indicators. These are things
like price history, volume, etc. These trades are very effective in the short
term, where trades can happen in terms of seconds or milliseconds. This is
where computers excel and humans fall short. This sort of trading is based
mostly on breadth. If you look up the fundamental law of active portfolio
management, you'll see that breadth is less effective (exponentially less so)
than skill is.

In the case of the long term, however, trading is done with fundamental
indicators. These things can be more or less intangible and have to do with
market events, people, and other indicators of company value that are hard to
translate into math. Using fundamental indicators for portfolio management is
what humans are better at, and these pay off in the long term (see Warren
Buffet). These trades are done with skill, which, as I stated earlier, is
exponentially more effective at creating gains than breadth.

In short, it takes a huge amount of breath to get the gains required by a
relatively small amount of skill. Computers are better by far at breadth,
while humans are better (for now) at skill. This, I think, is why humans still
trade.

~~~
wbsgrepit
Not to mention the spectrum between intuition/innuendo and flat out insider
trading knowledge that is cornerstone to the market -- trading algorithms
don't do well against this human factor.

------
spott
Information not in the market.

Stock traders trade on rumor, fact and everything in between. They (can) look
at who the company is run by, and how much they trust him, and look at the
people in upper management. They can look at and understand news.

Part of the difficulty of the stock market is that it isn't a closed system:
people make decisions to sell stocks based on the fact that they are poor this
month. Until computers can understand and process all the external data,
people will need to be in the trading loop.

The big exception to this is the trading the noise: high frequency traders
trade against each other, closing the spread in bid/ask prices. However, they
are essentially trading against other computers at that timescale, which makes
them pretty amenable to algorithms.

~~~
talmand
You reminded me of this scene from Trading Places:

Randolph Duke: Exactly why do you think the price of pork bellies is going to
keep going down, William?

Billy Ray Valentine: Okay, pork belly prices have been dropping all morning,
which means that everybody is waiting for it to hit rock bottom, so they can
buy low. Which means that the people who own the pork belly contracts are
saying, "Hey, we're losing all our damn money, and Christmas is around the
corner, and I ain't gonna have no money to buy my son the G.I. Joe with the
kung-fu grip! And my wife ain't gonna f... my wife ain't gonna make love to me
if I got no money!" So they're panicking right now, they're screaming "SELL!
SELL!" to get out before the price keeps dropping. They're panicking out there
right now, I can feel it.

Randolph Duke: [on the ticker machine, the price keeps dropping] He's right,
Mortimer! My God, look at it!

------
hsk
You're overestimating the effectiveness of computers in taking in data and
finding patterns. If you throw data naively at an algorithm, you'll get
garbage. It's especially difficult to make sense out of trading data because
of the sheer size and percentage of noise.

For any given trading strategy, a ton of thought, testing, and domain
knowledge goes into creating the algorithm. It is not a black box that writes
itself.

That said, computers are far more effective at certain tasks, especially
latency sensitive simple calculations, just as calculators are far better at
doing arithmetic.

~~~
ar7hur
I agree. And the OP is also underestimating the effectiveness of insider
trading.

------
chollida1
Well the truth is that for some firms it is for about 99.9% of all trades.

Renaissance Technologies founder Jim Simmons is famous for saying they didn't
override the algorithm.

In practice most HFT firms do a mix of both. The algos will do the vast bulk
of the trading but you have human traders monitoring algos to do clean ups for
cases where the algo gets "stuck". What defines "stuck" really depends on the
sophistication of the algo and the firm itself.

Some algo's, such as internalizers's for crossing bought flow are so simple
that there probably doesn't need to be much over site at all.

Market making is very similar, with the exception of a flash crash where they
might pull out, market making algos should just run themselves.

------
SEJeff
Well for starters, someone has to write the algorithms. Not everyone has vast
amounts of computing power to find the right inputs for the right algorithms
or genius level quants to write the math which coders turn into strategies.
There are so many small niches in the market that a human can still make a
reasonable living if they find an area not traded by others aka "making a
market". Now when it comes to competing with the machines, you pretty much
nailed it.

Source: I work in electronic trading and have the past ~7+ years.

~~~
Coding_Cat
Would you happen to know some good reads (either on- or off-line) for someone
interested in computer trading with a technical background (currently doing a
Msc. In Computational Science != Computer Science)?

I have been interested in doing a bit of electronic trading as a hobby (and
hopefully a small profit, but I know better than to hope for now) and possibly
a back-up career path. But while I've found enough information about trading
in general I found it rather hard to find anything dealing with automated
trading specifically beyond _very_ simple models.

~~~
walterbell
Try this blog and book, a rare mix of practical code and market analysis,
[http://qoppac.blogspot.com/p/systematic-trading-start-
here.h...](http://qoppac.blogspot.com/p/systematic-trading-start-here.html)

~~~
Coding_Cat
Thanks, seems to be some useful stuff in there.

------
mjwhansen
There are a lot of different schools of thought about approaching the stock
market. One of them is the "chart" approach, where you (or an algorithm) try
to discern the future movements of a company's stock price based on the past
performance. Sometimes this works. Mostly, though, it doesn't.

There's also a big difference between "trading" and "investing." Trading is
what you've described -- buying shares in the morning and hoping they'll go up
in the afternoon so you can sell them later. Investing is buying shares of the
company to become a part owner and hold them for years or decades, not days.

If you looked at NFLX's chart in 2012, you could "discern" that the share
price would continue to hover around that price, maybe go up a little, maybe
go down a little. And you could have bought it in September 2012 for $8 a
share and sold for a nice $1 profit in October 2012 for $9 a share (split
adjusted). But what the chart wouldn't have told you -- and would never have
been able to tell you -- is that it would skyrocket in 2013 and up to its
current split-adjusted price of $110 a share. The thing is, the chart never
would have told you about this. And even a "pure" numerical analysis like
could be done by a computer -- P/E ratios, cash flows, etc -- would not have
predicted that growth. You could do DCFs all day every day in 2012 and never
predict Netflix's rise. There are a lot of things that go into a company's
rise that aren't numerical, like the quality of management, market moat,
market growth, etc. And, of course, you had to buy it, and hold it for years,
in order to see that return.

(In the interests of full disclosure, I should probably note that I'm a bit
biased in this. I work for The Motley Fool, which advocates for long-term buy
and hold investing, and produce a podcast for growth investors called Rule
Breaker Investing.)

~~~
kspaans
Good point about trading versus investing. When index investing advocates say
that it's hard to beat the market over the long term, I've always wondered
what the difference is with traders. Do their returns beat the S&P500? Does an
individual who trades forex as their day job do better than stuffing their
money in an index fund for 40 years? Lately I've learned that larger traders
can provide value apart from buy&hold, e.g. market making, finding the right
price for something, shorting bubbles.

~~~
mjwhansen
I don't have specific numbers on traders, but I do know that 80% of mutual
funds underperform their benchmarks. One reason (of many) is that they're
under pressure to provide numbers every quarter, or to seem like they're
reacting when something short-term happens, so they sell out of positions that
would have been more profitable had they held onto them.

Index funds are a great solution for investors who want to invest passively
and get the market return (which averages 10% a year over the long term --
might be -20% and +30% year-over-year, though).

The thing with shorting bubbles is extremely difficult to get right, and
"finding the right price" doesn't necessarily mean you're making a good
decision. Amazon was trading around $300 a share earlier this year -- which
many trader types saw as outrageous -- but it has doubled this year alone.

I suppose, in general, I follow the Black Swan approach: humans are terrible
at predicting the future. Absolutely awful at it. But I'll take a "bet" on --
by which I mean, buy part of -- a company with great management/people and
solid financials (i.e., no debt, positive revenue growth, FCF positive).

------
leroy_masochist
Former investment banker here. This is a question I had when I hit the desk,
and it's an important question.

The short answer to your question is: because most of the volume traded on
exchanges is large blocks of stock being bought and sold by institutional
investors, and you need humans to make these deals happen.

Longer explanation as follows:

On the trading floor [0], you have two groups of people: the sales team and
the traders.

The sales team gets paid when they make markets, i.e., connect buyers and
sellers. Specifically, the financial institution takes a fee that's a very
small % of the overall transaction volume, and some of that goes into the
sales team's bonus pool. The more stock trades flow through the firm
(specifically, their business unit), the more they get paid.

The traders, on the other hand, gets paid to do two things, which are really
the same thing: a) not put too much of the firm's capital at risk and b) set
the firm up to make money by buying securities low and selling them high.
Every trader has a "P&L" (profit & loss) number, which is the total amount of
money they've made or lost for the firm since the start of the fiscal year.
They get paid a bonus based on this number. They tend to know exactly what
their number is at any given time.

So, there is actually a lot of tension on the trading floor between the sales
team and the traders, because the sales team wants a lot of volume to go
through their business unit, and any given trader wants to maximize her P&L.

Real world example might be: the sales dude gets a call from a hedge fund
saying, "we want to sell $100mm of our shares in Alphabet at $720". He then
shouts over to the trader (who sits close to him) to tell her about the call
and she thinks for a couple seconds [1] and then says, "you need to make a
market for 80mm of those shares at that price, I'm only taking 20mm."

In other words, the trader is saying that she'll only tie up $20mm of the
firm's capital on this particular trade [2]. The sales person might come back
and say, "c'mon, they took that $50mm of Microsoft stock you were trying to
get rid of last quarter, we as a firm owe them a favor" to which the trader
might respond, "OK, we'll take $30mm tops". So then the sales person will get
on the phone and start calling everyone (other mutual / hedge funds, pension
funds, etc) who might be interested in buying Alphabet at $720. Maybe the
sales person makes it happen; maybe they don't. In any case, they need to
figure out whether they can get 70 million dollars worth Alphabet stock pre-
sold to other people in the market at $720 before they get back to the hedge
fund trying to sell it with a response as to whether they can make the trade.

All of this involves MASSIVE HUMAN FACTORS. I'm sure we will one day be able
to train AI to work through various constructs of "we owe them a favor" but
right now you still need humans to get big trades like this done. And again,
big trades like this constitute the majority of the overall volume in the
market. So, that's why trades don't run entirely on algorithms...yet.

[0] I was in banking, not S&T, but have a decent understanding of how this
works.

[1] Being able to make decisions of this magnitude in a couple of seconds (and
have them be good ones) is one of the two skills you need to have as a trader;
the other one is not letting the outcome of the last trade (good or bad)
affect your thinking on the next trade.

[2] There is potential for both upside and downside in a decision like this;
if the stock appreciates, the firm can profit by selling the stock at a higher
price than it paid, but the reverse is also true. This is also an example of
why "proprietary trading" is such a blurry line. In order to make markets for
big trades, firms usually have to put their own capital at risk, even for a
few minutes. At what point are they trading for their own profit vs.
temporarily assuming risk in order to broker a deal between two
counterparties? Go read Matt Levine's archived columns at Bloomberg if you
find stuff like this interesting.

~~~
kcbanner
So once the pre-sell is agreed on are orders made on the open market? Or do
these deals take place outside of that?

~~~
leroy_masochist
No, the shares change hands between parties via the licensed broker-dealer.
However, the trade is reported to the exchange assuming the stock is listed.

The key here is that Big Mutual Fund Inc. doesn't want to announce to the
world that it's trying to sell a massive volume of shares; it wants to get the
deal arranged quietly so that other people in the market don't trade the stock
down in anticipation.

The price of the trade will be continually adjusted until the trade is
executed. Usually the firm has an approved range to work within. If the price
steps outside this range, many more phone calls will ensue.

~~~
kcbanner
I see, thanks!

------
brohee
Because ultimately, stock prices move not based on "the market" but on things
happening in the real world?

How does an algorithm interpret e.g. an Apple Keynote? By the time Twitter
sentiment analysis (if such thing is really useful) gives results, an human
trader already took a position...

~~~
rezistik
Humans are better at recognizing satire as well, April Fools jokes have
famously caused some algorithms to react poorly.

------
zrail
Why do I need an algorithm to find a good business, buy shares, and sit on
them forever?

------
Mikeb85
There are limits to algorithms. Stock prices are affected by human behaviour,
which can't always be predicted.

Not to mention the various macro variables, like wars, weather, crime, etc...
Plus, what time frame should the algorithms trade on? They're very good for
predicting the very short term, I haven't seen much evidence that they're good
for predicting longer time spans.

While algos eat up arbitrage and electronic brokers replace human ones, humans
are still very good at other forms of trading... Not to mention, a large part
of market activity isn't even trading - it's long term investing and
collecting dividends.

------
jackgavigan
It's impossible to create an algorithm that encompasses all the factors that
contribute to a company's stock price (or, perhaps more accurately, all the
factors that a given investor _believes_ will affect the stock price). There's
also an implementation gap between the model a stock analyst can create in
Excel and its implementation as an algorithm. The former is within the reach
of far more people than the latter.

Finally, don't forget that somebody has to design and write the algorithms.

------
chad_strategic
This is a pretty good question... Not sure if it will be solved here.

I used to be a value investor around 2000-2008. A value investor would be
something like Buffet or Peter Lynch. However I did make a lot money in Sept.
08 because I determined the market was over valued.

What I didn't forsee, was how much the dollars the Federal reserve would print
and inflate the economy.

Regardless, after that I built my own algorithm, because I no longer believe
in the structure of the market. I would rater trust numbers. Meaning there are
to many analyst pumping stocks, federal reserve, insider trading, spoofing
trades, ETFs, deratives, and financial warfare it's hard to make a true value
investment. Yes, I have read the buffet / Grahm books, but those are over ~60
old.

I think it is Virtu (electronic trading / hedge fund) that hasn't had a day
where they lost money since early 2009? I know Goldman and JP Morgan 90% of
the time trade every day for a profit. So a lot of the market is already
trading electronically. I think zerohedge.com has estimated the 70% of the
market trades on electronically and that article was few years ago.

It's funny, because I have devised methods using social media / programming to
manipulate the price of stocks. If I can think of ways to do that I'm sure
sure Wall St. already is doing it.

Anyways here my algorithm it tracks over 500 stocks: [http://www.strategic-
options.com/trade/](http://www.strategic-options.com/trade/)

~~~
nether
Your site charges $600/year. That's 6% of a $10,000 portfolio. Do you think it
can make a 6% gain in a year after taxes and inflation to cover its costs for
that size portfolio? 6% gain is about the long term average of the S&P500,
after inflation, so you're proposing doubling the market gain.

~~~
chad_strategic
Good question.

Let's not forget that S&P was down ~40% in 2008. So you are correct that long
term average is ~6%, but then you have years like that, kind of makes the long
term average meaningless. Don't forget your financial adviser took a cut on
your losses in 08 as well.

At $600 a year is pretty cheap in comparison to what other trading tools.
(news letters, chat rooms). This isn't necessarily for people looking a
retirement fund.

I'm also marketing something cheaper [http://www.strategic-
options.com/trade/alerts](http://www.strategic-options.com/trade/alerts) but
this is for people who trade semi-regularly. I'm not sure if you fit in that
marketing demographic. However, you can see from the website some stocks like
Amazon have return far greater amount than the $600 initial cost.

This new ETF / index strategy is nice, I working on a portfolio that would
only trade ETFs. That would charge only 1.25% of AUM. But that's still in the
works...

~~~
nether
So you're not really sure it could make up its cost?

~~~
chad_strategic
The particular service that I'm offering, the membership. It could surely make
up your cost... But it is more oriented towards do it yourself traders, it's
not necessarily geared for Buy and Hold. For example on 1/28/15 the algorithm
produced a buy signal on Amazon. If you followed that tip on 1/28 or 1/29,
then you would have received ~100% return your investment.
[http://www.strategic-options.com/trade/stock/amazoncom-
inc](http://www.strategic-options.com/trade/stock/amazoncom-inc)

So your $10,000 dollar invest would now be worth $20,000. That would surly
cover you membership cost of $600. You can find other winners here:
[http://www.strategic-
options.com/trade/open/market_analysis/...](http://www.strategic-
options.com/trade/open/market_analysis/winners) (click the winners tab)

I also have a Wiki/FAQ here: [http://www.strategic-
options.com/trade/open/wiki](http://www.strategic-options.com/trade/open/wiki)

If you have any other question feel free to email here chad (at) strategic-
option.com

------
anthony_barker
What is an algorithm in your definition?

If it includes where and how to place the trades (a smart order router) I
would say 99% of trades are managed by an Algo. Also there are the "dumb"
VWAP, POV, TWAP algos which represent the bulk of "smart" buyside money as
internally most firms use vwap as benchmark.

The bulk of retail orders in the US are on the other side of an algo from
citadel, knight or citi. And often buying 100 shares of MSFT at market only
really needs a decent SOR to provide BestEx.

Block trading still often gets fed to VWAP algos unless the stock is illiquid.

Finally the most interesting execution algos (implementation shortfall algos)
are hard to explain and only statistically outperform.

If you are talking about actual investing - the first question is which asset
class in which to park your money. If you can get an algo to do that (like
Renaissance) you will be rich.

------
dragontamer
Skill? I'm putting up trades as an investment. My money can't just sit around
in a bank account forever, it needs to be invested to make actual gains.

All the hedge-fund managers that claim to have algorithmic trading have
extremely high expense ratios. So its cheaper if I made trades myself.

I mean, its only $7 to execute a trade off of Scottrade or E-Trade. While
buying a mutual fund with algorithmic trading will cost you like 50 to 200
basis points per year.

Yeah, its cheaper to trade in the raw or to just buy SPY or Vanguard funds
(which are passively invested without algorithms)

------
simo7
The amount of trading done by machines decreases as your investing timeframe
increases.

So at the extreme (high frequency trading), all trading is indeed done by
machines.

------
codeonfire
Humans invest, computers trade. Its not about who is better at predicting.
Most companies will gain value over time. There's no prediction to it.
Algorithms are not predicting either. They are trying to fake out other
algorithms and profit off of market inefficiencies. You don't need an
algorithm, though, to buy something and then sell it six months later.

------
lujim
In addition to an algorithm not having all available information on the market
there is also the fact that any niche or inefficiency in the market can be
duplicated by others until it is negated.

If you discovered that the market always goes up on Tuesday and drops on
Wednesday that only works until everyone else discovers the same thing and
starts selling on Tuesday and buying on Wednesday.

------
oaktowner
Most stock brokerages spend lots of times gathering data from the companies
themselves -- the management group, the customers, industry analysts, etc.
They are not making investment (BUY/HOLD/SELL) recommendations based on the
past fluctuations of the market, rather on their "expert" appraisal of the
company's worth versus its current stock price.

------
blazespin
You mean the computers that can't even understand the sentence "We saw grand
canyon flying to chicago."

------
holri
I recommend to read Warren Buffet or his teacher Benjamin Graham. Buffet uses
a computer, but only for Bridge playing.

------
tmaly
market conditions are always changing as are regulations. In 2008, they
implemented an emergency short sale rule that banned short selling for a
select set of stocks. An algorithm would not know what that means, humans have
to step in to ensure stocks are not shorted during these times.

~~~
Peradine
Forgive my ignorance, but would it not be quite easy to tell the algorithm not
to short certain stocks?

~~~
devin_liu
You're asking the wrong question. That's just one case where the computer
doesn't understand the human side of things.

What you should be wondering is what are some cases where there is absolutely
no advantage to using a computer. Look up the story of "George Soros breaks
the bank of England." Or Hedge funds shorting Volkswagen when it was the most
expensive stock in the world. Only to find out that Porsche owns 75% of it.

------
danmaz74
Fortunately, not all investments are based purely on technical analysis for
short run gains.

~~~
VLM
Even the ones that are, are basically a poker game between at least two
intelligences, and AI is notoriously not that good at poker. There are plenty
of stories about human vs AI poker players to google for where the AI either
loses miserably or under weirdly limited circumstances merely loses
significantly.

The computers do beat human when the game is intentionally played too fast for
humans to physically / biochemically keep up (HFT) and humans usually destroy
algos for long term (admitted or not insider trading or line of business
expertise). The only really interesting timescale is the daytrader/poker
player.

------
oscarfr
The simple reason is that people make money by trading stocks. And they make
enough to keep doing it instead of doing something else.

It's simple economics. If they wouldn't be able to make money they probably
wouldn't trade.

~~~
vpribish
well, the popularity of gambling shows that there's more to it than that.
There's thrill, the perception of control, habit, and bad-math too.

------
anpk
I've been trying to find patterns ([http://newsp.in](http://newsp.in)). I dont
believe its an exact science yet, but its always fun to try.

------
artmageddon
How would an algorithm react to / speculate on the most recent news of, say,
the VW pollution-cheating fiasco? What about embargoes or war breaking out
between certain countries?

~~~
SEJeff
Using sentiment analysis. In fact, there are some traders (crazily!), which do
just that.

[http://www.wsj.com/articles/tweets-give-birds-eye-view-of-
st...](http://www.wsj.com/articles/tweets-give-birds-eye-view-of-
stocks-1436128047)

[http://tradethesentiment.com/](http://tradethesentiment.com/)

~~~
notahacker
An algorithm parsing the aggregate reaction of mostly non-experts, mostly to
news delivered over relatively slow mainstream media channels, is going to be
_much_ slower than a human hearing the investor briefing and hitting the "sell
at any price" button.

------
gorbachev
Because algorithms can't wine and dine potential investors to convince them
they're the best in the business for picking stocks.

~~~
hangars
Machiavellian. You are also overlooking pump and dump Twitter accounts which
sway stock by micro-increments, this tactic becoming more of an art than a
science these days (Twitter clamped down on those accounts in recent years).
Also see [http://www.huffingtonpost.com/2013/03/11/twitter-
hoax_n_2850...](http://www.huffingtonpost.com/2013/03/11/twitter-
hoax_n_2850892.html)

------
smrtinsert
They're not. Retail trading is a fools game.

------
whatok
[https://xkcd.com/1570/](https://xkcd.com/1570/)

------
kaa2102
Limited computing power, uncertainty and the false perception of perfect
information (or perfect intuition).

------
slantaclaus
Computers aren't normally so good at predicting the future?

------
blahblah3
Even if at some point humans no longer had any edge, by pure randomness some
of them would still have really good performance. So even in a perfectly
efficient market you wouldn't expect humans to be taken over.

------
lordnacho
I've been in the market for over a decade, and here's my take:

\- Lack of sophistication. "Classically" trained finance people don't know
much about computers. I took a finance class at a top business school, and
it's nothing compared to Engineering. A bit of time-value-of-money and maybe
some option math, but really it doesn't come close to the sophistication of a
CS or Engineering course. I went to a meeting last week with a guy who wanted
an automated trading system. He hadn't heard of Python. He didn't have any
idea how to execute other than on 3rd party programs (which of course use
algos, but he was just providing the decisions).

\- Lack of scale. There's a lot of family offices who have a few tens to
hundreds of millions of dollars. If they wanted an algo trading guy, they'd
have to pay him a lot of money, you'd want more than one, and you'd need
infrastructure. Plus there's the risk you get all this, hire the guys, and
their results are no better than random. A lot of small fortunes like this
tend to spend more time in tech-soft areas, like private equity or private
debt. The stock trades are an afterthought that they can't spend much resource
on.

\- Two kinds of decision making: arbitrage and investment. The put it bluntly,
arbitrage is easy to mechanize. If some guy quotes some options at the wrong
value, it's obvious you want to trade with him. There's looser arbs (things
that sort of always come back to normal), but the principle is the same. In
some sense, it's not a financial challenge, it's a technological one. For
investment (I think XYZ corp will go up), you need to have a sense of what
risk you want to take. Utility functions are not easy to put into code. You
can try, but you end up with situations where you decide not to have the algo
on. There's also the principal-agent problem; most traders are agents, they
need to look good to their boss. They need to be able to explain why they are
betting on some company. Often, more effort goes into how to justify your
trades than what trades to do.

\- Things that can't go into a machine: I worked with a guy who used to go
meet the CEOs, look them in the eye, and ask them if they'd make money. Now
I'm not saying this approach works, but if this is your investment edge, how
are you ever going to put that in a machine?

\- Insider information: taking this in the loose sense, not the criminal one.
If you're highly dependent on understanding some part of the market better
than others, you may be better off talking and networking rather than coding.
Goldmans are great at this. Every time you meet them, they offer a bit of info
in exchange for yours. It lets them see things like the mortgage bubble before
it happens, whereas a model would probably have issues due to the small amount
of computerized data.

