
Model beats Wall Street analysts in forecasting business financials - rbanffy
http://news.mit.edu/2019/model-beats-wall-street-forecasts-business-sales-1219
======
_sword
Hi former Wall Street enterprise software analyst here. Financial models were,
generally speaking, tightly managed by the companies being modeled. For each
upcoming quarter and full year, management (typically investor relations,
sometimes CFO) would hop on a call with my team and discuss where our
estimates were vs. consensus. Sometime, "consensus" was an internal measure
and not what was reported by e.g. Bloomberg or FactSet.

If estimates were particularly out of bounds from consensus, they would
politely ask how we modeled their business, if we would like help modeling
their company, if we had a particular reason for out of bounds estimates, etc.
That was a firmly worded but polite way to describe that the estimates might
need some review and adjustment.

~~~
hdevarajan
Were parties aware that this was close to or a de facto reg fd violation
([https://www.sec.gov/rules/final/33-7881.htm](https://www.sec.gov/rules/final/33-7881.htm))
and/or how was this handled internally?

~~~
wilg
Could you explain more? That is like a hundred pages of regulations and I
don't know anything about this.

~~~
OldManAndTheCpp
Short answer: reg fd requires companies to disclose information material to
investors to all investors. The previous post is suggesting that the close
work with the bank analysts is conveying material information without proper
disclosure. My hunch is that the legions is compliance lawyers at both the
banks and at the companies have deemed this to be within the bounds of the
regulation, but we’ll see if the SEC/US Attorneys agree.

~~~
oldgradstudent
Being cynical, I'd assume the "help" they offer is more likely to be stock
market manipulation than disclosure of material information.

~~~
nl
That's incorrect for these style of calls. The help is more akin to marketing
- the company is arguing that they are doing better than the analyst thinks,
and the analyst's company is offering their help to explain that to them and
to other analysts.

~~~
oldgradstudent
That's exactly what I meant. They are not necessarily disclosing real
information, they are trying to increase the stock price.

~~~
nl
That isn't what stock market manipulation[1] means.

    
    
      Market manipulation may involve techniques including:
        Spreading false or misleading information about a company;
        Engaging in a series of transactions to make a security appear more actively traded; and
        Rigging quotes, prices, or trades to make it look like there is more or less demand for a security than is the case.
    
    

Presenting real information in a way that makes it more clear to investors is
absolutely not market manipulation, and no interested party would ever claim
otherwise.

[1] [https://www.investor.gov/additional-resources/general-
resour...](https://www.investor.gov/additional-resources/general-
resources/glossary/market-manipulation)

------
hendzen
Citadel and Two Sigma both have had large teams working on this for years.
They are making a lot of money doing it but barriers to entry are very high.
You need to collect all the data in a way such that you can reconstruct how it
looked at any point in time. The vendors can’t be trusted to not make
retroactive corrections, so you have to collect it for years before it becomes
useful. Doing that takes a lot of time and money.

~~~
_sword
One of the wildest things I found while working on Wall Street was that ad
blockers are wholesale selling data to analysts and funds. For example, I
reached out to Ghostery asking if they had data about the data analytics tech
/ ad tech vendors used by industry, website, etc. to estimate market share and
they responded back with a sales person telling me I could buy that data from
them for five figures per year and that a number of my peers and clients were
doing the same.

~~~
stubish
Ghostery seems to have fallen completely out of favor for this behavior,
replaced by the EFF's Privacy Badger, actual ad-blockers, and even some
browser built in functionality.

It would be interesting to know which other privacy plugins that sell data.
Current recommendations seem to be pretty unanimously favoring plugins which
do not, but it is always a moving target.

~~~
pythux
Hi, I work on the Ghostery extension.

In addition to my answer regarding the business model of Ghostery above:
[https://news.ycombinator.com/item?id=21897809](https://news.ycombinator.com/item?id=21897809).
I'd like to add a few things to address your points:

(1) Ghostery did not "fall completely out of favor", as far as I can see. It
is true that in _some communities_ (in particular some sub-reddits), _some
people_ tend to recommend different addons instead (often not based on any
technical arguments, though), but this is not a trend that can be generalized
(more people continue to recommend Ghostery as a very solid privacy protection
suit).

(2) EFF's Privacy Badger is not a replacement for Ghostery, for more
information about why, we wrote about it in the past[1][2].

(3) Ghostery has an "actual ad-blocker" built-in, in fact, it is one of the
most efficient out there as was shown in a study that we published this
year[3]. The adblocker as well as benchmarks are open-source and anyone can
run them locally to verify the claims. We also more recently wrote extensively
about the internals of this adblocker[4].

References:

[1]:
[https://whotracks.me/blog/how_cliqz_antitracking_protects_us...](https://whotracks.me/blog/how_cliqz_antitracking_protects_users.html)

[2]: [https://0x65.dev/blog/2019-12-19/blocking-tracking-
without-b...](https://0x65.dev/blog/2019-12-19/blocking-tracking-without-
blocking-trackers.html)

[3]:
[https://whotracks.me/blog/adblockers_performance_study.html](https://whotracks.me/blog/adblockers_performance_study.html)

[4]: [https://0x65.dev/blog/2019-12-20/not-all-adblockers-are-
born...](https://0x65.dev/blog/2019-12-20/not-all-adblockers-are-born-
equal.html)

------
listenallyall
Like virtually every other attempt at predicting future events in complex
systems, equity analysts -- even very knowledgeable, hard-working, well-
connected, experienced ones -- are mostly just guessing. They regularly get
beaten by a random dartboard in the WSJ, just like the majority of mutual
funds trail behind index funds. Not much different than ESPN's former coaches
and players -- smart, successful, domain experts -- whose sports predictions
are not much better than random chance.

~~~
ses1984
You don't have to be much better than random chance to make mountains and
mountains of money.

~~~
listenallyall
You can actually be worse than random chance and still make mountains of
money.

The fact is, for most jobs like this, it is far more important to look & speak
the part -- to be able to make other people think you really know what you're
talking about -- than it is actually to be accurate. Most popular weather-
people are attractive and telegenic. Very few ever are actually measured by
accuracy of predictions. Similarly, equity analysts who can impress the client
base (who are, mostly, non-financial professionals) rarely get called out for
lack of accuracy. Not saying it never happens, but most of the time the
analyst (with the help of their employer) takes all the credit (for hard work,
knowledge, insight, etc) when they are right, and blame "unexpected
circumstances" or "unforeseen events" when wrong.

~~~
TeMPOraL
You can't be worse than random. If your estimates on a binary question are
consistently wrong, all you need to do is to do the inverse of what your
estimates tell you, and now you're better than random.

~~~
listenallyall
Irrelevant to this profession and discussion.

------
hnews_account_1
The entire sell side business is filled with sycophant analysts being paid
under the table for producing good reports. The entire training set is
bullshit if all you have as your "correct" dataset are these corrupt fuckers.

~~~
lainga
Do you, in so many words, concur with _sword's comment further up?

~~~
hnews_account_1
No. He's describing one side of the equation, which is companies forcing you
to produce a favourable report. That is almost common knowledge at this point
in the industry and he's absolutely spot on.

What I'm including beyond that is stuff like your brokerage arm asking you to
produce a favourable report so clients will buy more of the stock. That's just
one example. There are blatant and flagrant floutations of rules and
procedures in these markets.

One recent example is Goldman producing a Buy report on tesla during the same
week that their investment banking arm was placing shares on the market for
tesla. Ideally you'd want a complete blackout on any sell side reports if your
firm is placing the fucking shares, but goldman just didn't give a fuck and
got away with it for that quarter (far as I remember, I didn't follow up
beyond the next quarter).

~~~
_sword
I disagree in part with this. I had complete freedom to publish a report in
contradiction with the firm's interest, which is also backed up by regulation.
Investment banking and equity research on the sell-side is firmly divided by
compliance and I could never ever speak with my colleagues in investment
banking without the explicit supervision of my compliance department. That
said, unless I was extremely highly convicted in an idea, it was usually
better not to burn any bridges long term with the bankers or the companies.

Edit add-on: You're restricted on what you can publish while your firm is
doing banking business with the covered company. The restrictions are
stipulated by a mix of regulation and compliance departments. Typically during
an offering you either can't publish at all, or you can only discuss points
factually without changing your opinion or estimates. That however means that
if you publish a report and you're buy rated, even if you're publishing a
factual update on a stock you're effectively reiterating your rating. I'm not
sure of Goldman's specifics, but they know what they need to do to meet
research compliance requirements.

~~~
hnews_account_1
> but they know what they need to do to meet research compliance requirements.

Yes sir. And they were caught wrong footed. But I mean, the number of these
"misdemeanours" they get away with has always been pretty high so it isn't
surprising. They _almost_ got away with 1MDB for fuck's sakes.

I don't know where you worked, but I was buy side, so I had full freedom to
publish my own recommendations. However, my PM was a smart man and always
wanted me to get on a call with the sell side analyst if I was disagreeing
with him. But since I was very junior, a senior analyst used to lead the call
(and was already tuned into the stock and following my models on the side
because he was my mentor). Man the senior analyst (an ex sell side analyst
from goldman) used to grill the ever loving shit out of the sell side guy on
the call. Threw him charts from my report, charts from bloomberg / factset,
news links etc on email while he had him on the call.

Like 3-4 emails exchanged just when we were on the phone. Sell side guy was
also highly fucking experienced. Never fucking gave ground and admitted that
he was being forced by the brokerage to mark it a buy. Always a stalemate and
it was wondrous to me when I was new.

Then I understood they both knew it was going to be a stalemate long before
the call starts. My senior analyst is cursing the sell side guy to me, and the
sell side guy is probably cursing the two of us to his colleagues. We both
agreed to disagree, but he never could fully justify his position in my eyes.

~~~
_sword
We had total freedom to give any rating we could justify. We downgraded
banking names under coverage, or gave unfavorable ratings to some of the names
that bankers wanted us to add to coverage. These sometimes led to calls from
banking (moderated and monitored by compliance) demanding to know why we did
such a thing, or my director of research receiving emails from companies that
thought they deserved more favorable ratings that would personally insult my
team. We nevertheless made the right calls for our investors to the best of
our abilities, and I'll always stand by that.

I've been on a bunch of aggressive calls and they always get my pits sweaty. I
was on a few with the senior analyst I started working under that I really
remember. We had a contentious sell call on a stock and one of their top 10
holders wanted to speak with us. We hopped on the conference call and they
brought every one of their covering analysts, PM's, and even a trader onto the
call just to try and talk us out of our rating. It was one of those calls
where we knew neither side would find an agreement, but ultimately we both got
a better understanding of where our disagreements lay, which helped better
inform both sides about debates in the stock. The cursing and aggression from
these clients made that call especially tough.

On a different note, I found the former sell-side turned buy-side analysts to
be the toughest clients to work with. They know exactly how the research game
is played, but several seemed to have an especially large chip on their
shoulder and wanted to badger the sell-side guys since they had taken that
beating for years.

~~~
hnews_account_1
> On a different note, I found the former sell-side turned buy-side analysts
> to be the toughest clients to work with. They know exactly how the research
> game is played, but several seemed to have an especially large chip on their
> shoulder and wanted to badger the sell-side guys since they had taken that
> beating for years.

Very, very accurate from what I've seen. Especially from the former aggressive
sell side people like the senior analyst I worked with. They somehow get
redirected to the most aggressive current sell side people as well (despite
stock coverage being assigned however).

My coverage finally didn't have too many contentious names (I took up
defensives in fucking 2014 thinking the bull market was ending), but the ones
that did had very mellow sell side guys. Whereas the sell side guys handling
big tech and consumer discretionary names were aggressive as fuck. For
instance, the same senior analyst took like 2-3 hours reading up on my stock
before a sell side call, but with the consumer discretionary guy, he used to
read up from the night before because (a) the stocks were more complicated,
and (b) the sell side guys for these stocks were always super charged dudes
with a ton of info directly at their fingertips. You needed to adjust your
entire demeanour to deal with them differently.

------
mattrp
What really bothers me about these articles is that the authors claim to beat
but they only study 37 companies. What about the 38th? Or the 380th. I
guarantee you I can cherry pick 37 companies where a linear regression will
accurately predict earnings quarter over quarter... what does that prove?

~~~
mzz80
A study never proves anything. A p-value only tells you how likely the results
are by random chance. Presumably, the companies were chosen beforehand. It is
unlikely to have happened by chance that it beats for 37 companies. That’s
different then selecting the companies after the results.

------
Mikeb85
At the end of the day, securities pricing is based on supply/demand for the
security. Predicting precise sales numbers isn't that important, stock prices
move based on a plethora of other factors. How many times have we seen a
company out-perform expectations and then the stock plummet, or a company
under-perform and rally? It's cool that they're able to do this but it's not
that big of a deal...

~~~
axlee
> How many times have we seen a company out-perform expectations and then the
> stock plummet, or a company under-perform and rally?

Far, far, far less than a company's stock surging after a good quarterly, a
great product launch or tanking after a bad earnings report. You are
nitpicking.

Yes, outliers and "irrational" market reactions exist, but there is an obvious
arbitrage opportunity in deriving accurate sales numbers or other indices
before the market gets its hand on them.

~~~
AznHisoka
It is uncommon for a company to outperform expectations and plummet but very
common for it to simply _not react_ or dip slightly. Thus making it hard to
make any serious money if you can predict outperformance

However it is more common for a stock to tank if it misses expectations. If
you can predict underperformance, you can make some serious bank.

------
xiphias2
It's not the model, it's credit card data.

VISA / Mastercard could have its own hedgefund making great amount of money if
it didn't sell the data to other companies.

~~~
corporate_shi11
Sell the shovels!

~~~
xiphias2
Maybe you are right, but having a monopoly on the data makes competition
smaller and extracting revenue easier.

~~~
ta999999171
Their current position (apparently) keep them safe from antitrust.

------
anthony_doan
Small data, Kalman filter, belief...

They're using Bayesian statistic. Probably added some expert belief to their
prior distribution (informative prior). The whole article made it sound
spicier than it has to be. The majority of the ML algorithms out there have
too many parameters so it require a lot of data to overcome certain weaknesses
like selection bias in tree ensemble, etc... So with small data it seems like
ML are turning toward Bayesian. Also article may be missusing the term
inference.

~~~
starwars11
they do not use priors or expert belief. they solve the system id problem
directly - which in most problems is unresolvable

~~~
yalph
What does solving system id mean?

~~~
starwars11
estimating model parameters. normally people use heuristics because it’s an
unreasonable problem for linear dynamical systems without additional
assumptions. here, they’re able to produce analytical results and finite
sample analysis

~~~
starwars11
*unresolvable

------
floki999
The headline itself demonstrates ignorance as to what matters to an investor
putting money on the table. 1\. Sell-side analyst forecasts do not drive the
investment decisions of the more sophisticated investors. The sell-side is
just a big marketing machine and the value-add of the analysis is very low at
an individual stock level. 2\. The model ‘beats’ the consensus forecast on a
limited sample of names. By definition, the consensus forecast is an averaging
out which leads to dilution of any one analyst’s alpha - hence it is not an
appropriate benchmark.

It is a naive approach and study, but typical of academics who unfortunately
have little exposure to real-world investing/trading strategies.

The use of ‘alternative data’ is not new and is definitely leading to alpha
generation for some firms, but as mentioned by others such data-driven
strategies will usually have limited shelf-life.

------
airstrike
99,999 other models failed.

Let's see how well this model performs for n > 30 or across diverse
industries.

------
deepnotderp
Lots of comments about alternative data like credit card data.

But:

1\. There are many factors that influence the movement of a security. "Short-
term reality" (important KPIs basically) is just one element in the vector
that determines price.

2\. Getting long term exclusive alternative data is very difficult. Datasets
are rapidly commodified.

------
WomanCanCode
Great. Now fire all those business analysts and replace them with software
programmers.

------
aabhay
Don’t tell me what’s in your model, just show me what’s in your portfolio..

