
Andreessen-Horowitz craps on “AI” startups from a great height - dostoevsky
https://scottlocklin.wordpress.com/2020/02/21/andreesen-horowitz-craps-on-ai-startups-from-a-great-height/
======
m0zg
"Huge compute bills" usually come from training, or to be more precise,
hyperparameter search that's required before you find a model that works well.
You could also fail to find such a model, but that's another discussion.

So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's)
worth of cash on finding such models for your startup if you insist on helping
Jeff Bezos to wipe his tears with crisp hundred dollar bills. That's if you
know what you're doing of course. Literally unlimited amounts could be spent
if you don't. Or you could do the same for a fraction of the cost by stuffing
a rack in your office with consumer grade 2080ti's. Just don't call it a
"datacenter" or NVIDIA will have a stroke. Is that too much money? Not in most
typical cases, I'd think. If the competitive advantage of what you're doing
with DL does not offset the cost of 2 meatspace FTEs, you're doing it wrong.

That, once again, assumes that you know what you're doing, and aren't doing
deep learning for the sake of deep learning.

Also, if your startup is venture funded, AWS will give you $100K in credit,
hoping that you waste it by misconfiguring your instances and not paying
attention to their extremely opaque billing (which is what most of their
startup customers proceed to doing pretty much straight away). If you do not
make these mistakes, that $100K will last for some time, after which you could
build out the aforementioned rack full of 2080ti's on prem.

~~~
bob1029
I find it fun how the cost of the cloud is forcing people to consider what
absolutely must run in the cloud (presumably for stability and compliance
reasons) and what can be brought back on-prem.

We don't train ML models, but we are in a similar boat regarding cloud compute
costs. Building our solutions for our clients is a compute-heavy task which is
getting expensive in the cloud. We are considering options such as building
commodity threadripper rigs, throwing them in various developers' (home)
offices, installing a VPN client on each and then attaching as build agents to
our AWS-hosted jenkins instance. In this configuration we could drop down to a
t3a.micro for Jenkins and still see much faster builds. The reduction in
iteration time over a month would easily pay for the new hardware. An obvious
next step up from this is to do proper colocation, but I am of a mindset that
if I have to start racking servers I am bringing 100% of our infrastructure
out of the cloud.

~~~
blt
If I worked from home and my employer asked me to install a server in my home,
I would tell them to go fuck themselves.

It's noisy, it takes up space, and presumably I'm on call to fix it if it
breaks.

You should pay them an extra 24x(PSU wattage)x(peak $/Wh in area) per day for
the electricity too.

I'm alarmed that someone in your company felt this idea was appropriate enough
to propose.

~~~
bob1029
We would certainly compensate employees. I didn't feel it appropriate to
disclose every last detail regarding the arrangement in a thread which is only
tangentially-related to the OP.

This was my idea. I am a developer in my company. We are a flat structure. We
have a lot of respect for each other. I am on a standup with the CEO every
day. We all believe in our product and would happily participate in whatever
activity brings it to market more quickly. We do not hire or retain the kind
of talent that would flatly refuse to participate in experimental projects
like this. At least not without some sort of initial conversation about why
it's not a good fit for a particular individual.

I certainly see how someone might share your perspective. I used to work for a
souless megacorp and I could have easily found myself telling my former
employer to "go fuck themselves" if a proposal similar to this was imposed
upon me.

~~~
starfallg
Try the value data center providers like he.net.

A 42u rack and 1 Gbps connection is $400 per month.

Put cheap supermicro EPYC servers in rather than threadrippers (or build your
own). High capacity RDIMMs are cheaper than UDIMMs.

This will give you a much more maintainable solution than workstations at
employee's home via overlay VPN.

There is a maintenance cost for infrastructure that people tend to forget
these days.

------
shoo
> most people haven’t figured out that ML oriented processes almost never
> scale like a simpler application would. You will be confronted with the same
> problem as using SAP; there is a ton of work done up front; all of it
> custom. I’ll go out on a limb and assert that most of the up front data
> pipelining and organizational changes which allow for [ML to be used
> operationally by an org] are probably more valuable than the actual machine
> learning piece.

Strong agreement from me: I've never worked on deploying ML models, but have
worked on deploying operations-research type automated decision systems that
have somewhat similar data requirements. Most of the work is client org
specific in terms of setting up the human & machine processes to define a data
pipeline to provide input and consume output of the clever little black box. A
lot of this is super idiosyncratic & non repeatable between different client
deployments.

~~~
izendejas
That's because, ML and operations-research problems can be simplified to set
of optimization problems and the underlying math and statistics are all very
similar if not identical in some cases.

And the input matters, a lot. So the differentiating factor isn't the models,
it's the data and companies like Google figured it out a long time ago.

In short, find interesting problems, then the solutions -- not the other way
around.

~~~
killjoywashere
"The data" means more than pure computer science people want to admit. In any
"advanced" application, that means annotators. Radiologists drawing circles
around cancer, attorneys labeling contract clauses as unacceptable, drivers
labeling stop signs, etc.

ML is a mining problem. Digitizers are the miners. Annotators are the
refiners.

~~~
joe_the_user
Basically, the system is massively ad-hoc and driven by this large scale
annotation, training and testing.

The big question here is, _what happens when the world changes next year?_ You
rebuild the application. I know there are companies that advertise doing
continuous updating of deep learning models but it seems like calculating
total costs and total benefits is going to be hard here.

~~~
killjoywashere
Sometimes the mine makes money, sometimes it doesn't make sense to run the
mine.

~~~
moandcompany
To extend the mining metaphor, and relate back to the original articles:

People and organizations are chasing what they believe, or are told to
believe, is pay dirt.

Many unfamiliar investors have rushed in, possibly fearing missing out, and
fund many of the prospectors, yet many of the prospectors and investors aren't
really aware of the costs of running a mine, nor the practices required to run
them efficiently.

It turns out that there's more aspects to the value creation process than
dig/refine/polish (data/train/predict), especially when usefulness in
application matters and there are finite resources available for digging.

Companies selling shovels are some of the primary beneficiaries of this, by
selling shovels (i.e. renting compute) funded by the malinvestment.

Additional beneficiaries are the refiners (training experts) that are able to
charge steep labor premiums, however organizations are starting to figure out
that their refiners are expensive to keep idle and often operate the mines
poorly in terms of throughput/cost-effectiveness/repeatability/application
(see the various threads on "Data Engineers")

------
joshuaellinger
I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to
a $1K/mo hosting cost.

But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a
couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern
U.2 SSDs (say 500MB/sec per device).

I figure it is going to take me about $10K in labor to move and then $1K/mo to
maintain and pay for services that are bundled in the cloud. And because I
have all this dedicated hardware, I don't have to mess around with
docker/k8s/etc.

It's not really a big data problem but it shows the ROI on owning your own
hardware. If you need 100 servers for one day per month, the cloud is amazing.
But I do a bunch of resampling, simple models, and interactive BI type stuff,
so co-loc wins easily.

~~~
wpietri
I'm sure your right for your case. But I'd add one caveat for those less
experienced: if you own the hardware, you need to be prepared to go to the
colo when something breaks. The various clouds are a much nicer experience
when hardware fails. At the very least people should have enough spare
capacity that a hardware failure means going sometime in the next couple of
weeks, rather than getting up at 3 am and fixing things under pressure.

~~~
latch
Or take the middle road and just get rent the hardware (aka, dedicated
hosting). You pay more than colo but still way less than cloud, get the same
level of hardware support as a cloud provider but the same performance as
colo.

~~~
kavalg
Yep, for example Hetzner offers bare metal servers as well as cloud instances
at a very reasonable price. (Not affiliated in any way. Just a happy
customer.)

------
raiyu
The number of places where machine learning can be used effectively from both
a cost perspective and a return perspective are small. They are usually
tremendously large datasets at gigantic companies, and they probably have to
build in house expertise because it's hard to package this up into a product
and resell it for various industries, datasets, etc.

Certainly something like autonomous driving needs machine learning to
function, but again, these are going to be owned by large corporations, and
even when a startup is successful, it's really about the layered technology
on-top of machine learning that makes it interesting.

It's kind of like what Kelsey Hightower said about Kubernetes. It's
interesting and great, but what will really matter is what service you put on
top of it, so much so that whether you use Kubernetes becomes irrelevant.

So I think companies that are focusing on a specific problem, providing that
value added service, building it through machine learning, can be successful.
While just broadly deploying machine learning as a platform in and of itself
can be very challenging.

And I think the autonomous driving space is a great example of that. They are
building a value added service in a particular vertical, with tremendous
investment, progress, and potentially life changing tech down the road. But as
a consumer it's really the autonomous driving that is interesting, not whether
they are using AI/machine learning to get there.

~~~
Q6T46nT668w6i3m
How would you explain the rise (and success) of machine learning in science? A
lab that uses some learning-based method will likely be limited to just one or
two people (responsible for data acquisition, feature engineering, evaluation,
etc.) and extremely finite data.

~~~
ska
It's not clear there has been any deep impact actually, but there has been a
lot of discussion (and grant proposals)

I've seen a lot of cross pollination of ML and AI techniques into various
disciplines. A large percentage just didn't work at all, most of the rest were
more "kind of interesting, but". Nothing earthshaking happened although pop
sci press likes to talk about it a lot.

If you have more digital data than you used to, using modern free frameworks
and toolkits to do basic (i.e. older, boring, but understood) ML stuff to
understand it seems to have a reasonable return. Mostly I think this is
because it becomes accessible to someone without much background in the area,
and you can do reasonable things without having to put 6 months of reading and
implementing together before starting.

------
inthewoods
Having briefly worked for an AI company, I agree with the conclusion that AI
companies are more like services businesses than software companies. I would
add only one other thing: to me going forward there likely won't be "AI
companies" \- AI exists to power applications. And in my experience, unless
the output is truly differentiated, customers aren't willing to spend more for
something "powered by AI" \- they just expect that software has evolved to
provide the kind of insights that AI sometimes deliver.

~~~
shoo
For an example of a genuine software company vaguely in this ecosystem,
consider companies that build the tools that some AI/ML/optimisation systems
use as building blocks. Eg optimisation algorithms.

If you need to solve gnarly industrial scale mixed integer combinatorial
optimisation problems in the guts of your ML / optimisation engine, the
commercial MIP solvers (gurobi , CPLEX ) or non-MIP based alternative
combinatorial optimisation systems (localsolver ) can often give more optimal
results in exponentially less running time than free open source alternatives.

1% more optimal solutions might translate into 1% more net profit for the
entire org if you've gone whole hog and are trying to systematically profit
optimise the entire business, so depending on the scale of the org it might be
an easy business case to invest a few million dollars to set this system in
place.

Annual server licenses for this commerical MIP solver software was 0(100k) /
yr per server & the companies that build these products bake a lot of clever
tricks from academia into these products that you can exploit by paying the
license fee. ( my knowledge of pricing is out of date by about 7 years ) .

~~~
MrK93
I'm all for linear optimization and other optimization techniques. It's
refreshing to see other people talk about Gurobi, CPLEX, etc... Having done
research in the field of scheduling and now getting contacted by companies,
it's demoralizing to see that everybody usually speaks about machine learning
while many problems can be solved in a more precise way with other techniques.

------
rossdavidh
So, way back in the last millenium, I did my Master's thesis (way smaller deal
than a Ph.D. thesis) on neural networks. Since then, I have looked in on it
every few years. I think they're cool, I like using them, and writing multi-
level backpropagation neural networks used to be one of the first things I'd
do in a new language, just to get a feel for how it worked (until pytorch came
along and I decided for the first time that using their library was easier
than writing my own).

So, it's not like I dislike ML. But, saying an investment is an "AI" startup,
ought to be like saying it's a python startup, or saying it's a postgres
startup. That ought not to be something you tell people as a defining
characteristic of what you do, not because it's a secret but rather because
it's not that important to your odds of success. If you used a different
language and database, you would probably have about the same odds of success,
because it depends more on how well you understand the problem space, and how
well you architect your software.

Linear models or other more traditional statistical models can often perform
just as well as DL or any other neural network, for the same reason that when
you look at a kaggle leaderboard, the difference between the leaders is
usually not that big after a while. The limiting factor is in the data, and
how well you have transformed/categorized that data, and all the different
methods of ML that get thrown at it all end up with similar looking levels of
accuracy.

There used to be a saying: "If you don't know how to do it, you don't know how
to do it with a computer." AI boosters sometimes sound as if they are
suggesting that this is no longer true. They're incorrect. ML is, absolutely,
a technique that a good programmer should know about, and may sometimes wish
to use, kind of like knowing how a state machine works. It makes no great deal
of difference to how likely a business is to succeed.

~~~
jedberg
Saying that you're going to "use AI" is more akin to saying "we're going to
have a web application" back in 1998.

Back then a lot of startups didn't have websites, because they were making
other products (hardware, boxed software, etc). If they had a website it was
just a marketing page.

So saying that you were going to make a "web application" did in fact
differentiate you, in that it showed your approach was very different from the
boxed software folks, but it didn't tell you much beyond that.

~~~
all_blue_chucks
"Web application" came later. In the nineties it was called a "cgi web page"
by your webmaster.

~~~
perl4ever
In the nineties, there was a huge difference between 1995 and 1998. It wasn't
all that apparent to some of us until later, but things moved really fast in
that timeframe. The years leading up to 2000 were almost like the imagining of
approaching an event horizon or asymptote.

~~~
edw
What you’re describing is so hard to convey to people. In 1994 the we were
building raised-floor data centers with halon for suppressors and marveling at
our 2GB behemoth UNIX boxes. And writing our own web application framework
using CGI. In ‘99 we were renting a suite at a colo and putting our own
hardware there, running ColdFusion web apps. In ‘04 we were renting half a
rack at the same colo and trying not to write three tier Java servlet based
apps with 1,000 line web.xml files. And then AWS happened.

~~~
rbinv
CGI to ColdFusion to Java servlets. Sounds enterprise-y.

~~~
edw
It was all very start-uppy. What were you using to build your commercial web
applications in 1996 if not CGI? Mod_perl did not even exist until 1995, and
FastCGI didn't exist IIRC until after Netscape released their enterprise
server.

~~~
rbinv
Huh. I agree with CGI, but CF certainly had alternatives.

~~~
edw
The boring low-risk unsexy thing that works is often underrated. I didn't
choose CF — at the time I argued that it was a tool for scrubs, but the VPE
said, "Hey, I know it, and I know it can do the job." We launched that CF-
based web site on time and sold the company for $350MM fewer than six months
later. Only then did we incrementally port it to Java.

------
ativzzz
I agree with the author's opinion about

> I’ll go out on a limb and assert that most of the up front data pipelining
> and organizational changes which allow for it are probably more valuable
> than the actual machine learning piece.

Especially at non-tech companies with outdated internal technology. I've
consulted at one of these and the biggest wins from the project (I left before
the whole thing finished unfortunately) were overall improvements to the
internal data pipeline, such as standardization and consolidation of similar
or identical data from different business units.

~~~
jotakami
I was a consultant at one of the giant outsourcers and nod my head vigorously
at this comment. The least sexy projects were MDM (master data management) but
they were absolutely essential to the success of any other fancy
analytics/BI/ML project.

~~~
2sk21
Interestingly I too worked on MDM systems about ten years ago, when I was at
IBM Research. Ironically, one of my first ideas for applying machine learning
was in de-duplication of data in an MDM server. However the technology was a
bit too primitive back in 2010 and the project was a hard sell so it was
abandoned.

------
correlator
No need to look at AZ for this. If you're building "AI" I wish you a speedy
road to being acquired by a company that can put it to use. You've become a
high priced recruiting firm.

If you're solving a real problem and use ML in service of solving that
problem, then you've got a great moat....happy trusting customers.

It's not complicated

~~~
motohagiography
Sssh! Valuations are a function of projected market size and opacity of the
problem. Clarity like this collapses the uncertainty and destroys value. If
you pour enough capital into rooms full of PhD's something's gotta hit.

My way of saying, you're very, very right.

------
seibelj
I wrote an article I published a week ago about how AI is the biggest misnomer
in tech history [https://medium.com/@seibelj/the-artificial-intelligence-
scam...](https://medium.com/@seibelj/the-artificial-intelligence-scam-is-
imploding-34b156c3537e)

I wrote it to be tongue-in-cheek in a ranting style, but essentially "AI"
businesses and the technology underpinning it are not the silver bullet the
media and marketing hype has made it out to be. The linked article about a16z
shows how AI is the same story everywhere - enormous capital to get the data
and engineers to automate, but even the "good" AI still gets it wrong much of
the time, necessitating endless edge-cases, human intervention, and eventually
it's a giant ball of poorly-understand and impossible to maintain pipelines
that don't even provide a better result than a few humans with a spreadsheet.

~~~
scottlocklin
Coming from a fellow masshole: that's a great rant.

There was this meme in the 70s about "self driving cars" following magnetic
strips in the road in restricted highways. I remember at the time, being, like
8 and thinking "sure seems like an overly complicated train."

~~~
seibelj
Thanks man! Lifelong masshole here.

Your post was much better than mine, but I appreciate the comment.

------
harias
>That’s right; that’s why a lone wolf like me, or a small team can do as good
or better a job than some firm with 100x the head count and 100m in VC
backing.

goes on to say

>I agree, but the hockey stick required for VC backing, and _the army of
Ph.D.s required to make it work_ doesn’t really mix well with those limited
domains, which have a limited market.

Choose one?

Also assumes running your own data center to be easy. Some people don't want
to be up 24x7 monitoring their data center or to buy hardware to accommodate
the rare 10 minute peaks in usage.

~~~
jjeaff
>rare 10 minute peaks

But is that really the use case here? I haven't worked in ML. But I'm not
seeing where you are going to need to handle a 10 minute spike that requires a
whole datacenter.

A month's worth of a quad gpu instance on AWS could pay for a server with
similar capacity in a few months of usage.

And hardware is pretty resilient these days. Especially if you co-locate it in
a datacenter that handles all the internet and power up time for you. And when
something does go wrong, they offer "magic hands" service to go swap out
hardware for you. Colocation is surprisingly cheap. As is leasing 'managed'
equipment.

------
moab
I found it fun to read this after reading this other post that made the rounds
today about AI automating most programming work and making program
optimization irrelevant: [https://bartoszmilewski.com/2020/02/24/math-is-your-
insuranc...](https://bartoszmilewski.com/2020/02/24/math-is-your-insurance-
policy/)

------
dang
A thread about the original article, from a few days ago:
[https://news.ycombinator.com/item?id=22352750](https://news.ycombinator.com/item?id=22352750)

------
fxtentacle
I predict a great future for startups that sell pickaxes, err, tools for AI.

AI is like the new gold rush. And just like back then, it's not the gold
diggers that will get rich.

"Most people in AI forget that the hardest part of building a new AI solution
or product is not the AI or algorithms — it’s the data collection and
labeling."

[https://medium.com/startup-grind/fueling-the-ai-gold-
rush-7a...](https://medium.com/startup-grind/fueling-the-ai-gold-
rush-7ae438505bc2)

(from 2017)

~~~
moksly
Is it the new gold rush though. I work in a large organisation that has a lot
of data and inefficient processes, and we haven’t bought anything.

It hasn’t been for a lack of trying. We’ve had everyone from IBM and Microsoft
to small local AI startup try to sell us their magic, but no one has come up
with anything meaningful to do with our data that our analysis department
isn’t already doing without ML/AI. I guess we could replace some of our
analysis department with ML/AI, but working with data is only part of what
they do, explaining the data and helping our leadership make sound decisions
is their primary function, and it’s kind of hard for ML/AI to do that (trust
me).

What we have learned though, is that even though we have a truck load of data,
we can’t actually use it unless we have someone on deck who actually
understands it. IBM had a run at it, and they couldn’t get their algorithms to
understand anything, not even when we tried to help them. I mean, they did
come up with some basic models that their machine spotted/learned by itself by
trawling through our data, but nothing we didn’t already have. Because even
though we have a lot of data, the quality of it is absolute shite. Which is
anecdotal, but it’s terrible because it was generated by thousand of human
employees over 40 years, and even though I’m guessing, I doubt we’re unique in
that aspect.

We’ll continue to do various proof of concepts and listen to what suppliers
have to say, but I fully expect most of it to go the way Blockchain did which
is where we never actually find a use for it.

With a gold rush, you kind of need the nuggets of gold to sell, and I’m just
not seeing that with ML/AI. At least no yet.

------
whoisjuan
An many times all these AI computations go into solving mundane problems like
"What's the likelihood of this Ad to perform well".

AI is so shiny that makes people want to jump as fast as they can into that
boat but a reasonable objective analysis shows that a huge and not
insignificant amount of software problems can still be solved without relying
on the "AI black box".

------
DrNuke
You all know a GTX 1070 with 8GB on a gaming laptop with 32GB is still doing
wonders and covering 90%+ business cases when coupled with smart & batch
techniques the likes of you learn from fast.ai or under direct pytorch
implementation, right??

------
_bxg1
> Training a single AI model can cost hundreds of thousands of dollars (or
> more) in compute resources

Why don't they buy their own hardware for this part? The training process
doesn't need to be auto-scalable or failure-resistant or distributed across
the world. The value proposition of cloud hosting doesn't seem to make sense
here. Surely at this price the answer isn't just "it's more convenient"?

~~~
KaiserPro
because you are trading speed for cash.

Say you have $8M in funding, and you need to train a model to do x

You can either:

a) gain access to a system that scale ondemand and allows instant, actionable
results.

b) hire a infrastructure person, someone to write a K8s deployment system.
Another person to come in a throw that all away. Another person to negotiate
and buy the hardware, and another to install it.

Option b is can be the cheapest in the long term, but it carries the most risk
of failing before you've even trained a single model. It also costs time, and
if speed to market is your thing, then you're shit out of luck.

~~~
_bxg1
Why in the world do you need a Kubernetes deployment system to run a single,
manual, one-time (or a handful of times), high-compute job?

~~~
dsl
Because when all you have is a hammer, everything looks like a nail.

We have become so DevOps and cloud dependent that everyone has forgotten how
to just run big systems cheaply and efficiently.

------
jotakami
> Better user interfaces are sorely underappreciated.

This is why I’m much more excited by AR and VR than AI. Human brains are
fucking amazing at certain kinds of data processing and inference and pretty
mediocre at others. We should be focusing more on creating interfaces and data
visualizations that unlock that superpower for wider applications.

------
dcl
I'm not terribly convinced of point 4.

> Machine learning will be most productive inside large organizations that
> have data and process inefficiencies.

I strongly believe ML is at worst dangerous and at best pointless here. Data
and Process inefficiencies => garbage in, garbage out. ML is NOT a silver
bullet in large organisations that have these issues*, I've seen managers try
to adopt ML to solve issues, but the results are almost always suspect and/or
marginally better than simple if else rules but require a multiple people or
teams to get all the data and models right.

------
aj7
“ Embrace services. There are huge opportunities to meet the market where it
stands. That may mean offering a full-stack translation service rather than
translation software or running a taxi service rather than selling self-
driving cars. Building hybrid businesses is harder than pure software, but
this approach can provide deep insight into customer needs and yield fast-
growing, market-defining companies. Services can also be a great tool to
kickstart a company’s go-to-market engine – see this post for more on this –
especially when selling complex and/or brand new technology. The key is pursue
one strategy in a committed way, rather than supporting both software and
services customers.”

Exactly wrong and contradicts most of the thesis of the article - that AI
often fails to achieve acceptable models because of the individuality,
finickiness, edge cases, and human involvement needed to process customer data
sets.

The key to profitability is for AI to be a component in a proprietary software
package, where the VENDOR studies, determines, and limits the data sets and
PRESCRIBES this to the customer, choosing applications many customers agree
upon. Edge cases and cat-guacamole situations are detected and ejected, and
the AI forms a smaller, but critical efficiency enhancing component of a
larger system.

~~~
TheOtherHobbes
The thesis of the article is that this is going to be called consultancy.

Single-focus disruptors bad. Generic consultancy good - with ML secret sauce,
possibly helped by hired specialist human insight.

Companies that can make this work will kill it. Companies that can't will be
killed.

It's going to be IBM, Oracle, SAP, etc all over again. Within 10 years there
will be a dominant monopolistic player in the ML space. It will be selling
corporate ML-as-a-service, doing all of that hard data wrangling and model
building etc and setting it up for clients as a packaged service using its own
economies of scale and "top sales talent" (it says here).

That's where the big big big big money will be. Not in individual specialist
"We ML'd your pizza order/pet food/music choices/bicycle route to work"
startups.

Amazon, Google, MS, and maybe the twitching remnants of IBM will be fighting
it out in this space. But it's possible they'll get their lunch money stolen
by a hungry startup, perhaps in collaboration with someone like McKinsey, or
an investment bank, or a quant house with ambitions.

5-10 years after that customisable industrial-grade ML will start trickling
down to the personal level. But it will probably have been superseded by
primitive AGI by then, which makes prediction difficult - especially about
that future.

~~~
wayoutthere
The big consulting firms have been building in-house ML libraries for common
business problems for 3+ years. They don't need to acquire the data startups
because as the article points out, these models are commoditized pretty
quickly (especially when you have access to the transactional data of many
large multinational companies). There is no secret sauce to ML that makes you
any more likely to succeed with it than Accenture -- and they have a much
deeper pipeline than you do. ML is a mature capability at all of the
enterprise-tier consultancies, and they bundle it with their $100M system
deployments. The mid-market consultancies are working on it. There is very
little money to squeeze out of this market.

We're also a long way off from AGI. Nobody really even has a roadmap to what
an AGI would look like. Heck, DNN/ML techniques have been widely-known since
the early 90s; they just became practical with access to cloud-scale hardware,
so the current situation has been 25+ years in the making.

------
yogrish
Now a days DL models are becoming commodities very fast. By the time you train
NN to solve a particular problem, a new efficient model is out somewhere and
is available public. So you need to go through the process entirely or else
you risk losing business. Unless your NN is so unique like you are
handcrafting your own in which case you take lot of time to arrive at a best
model and you need more PhDs.

~~~
jeremysalwen
Props to the ML community for being so open.

~~~
fncypants
Open does not mean patent-free.

------
leetrout
That is a great write up and very accurate description of both the costs and
human intervention based on my experience with “AI” tools.

------
dvfjsdhgfv
> In the old days of on-premise software, delivering a product meant stamping
> out and shipping physical media – the cost of running the software, whether
> on servers or desktops, was borne by the buyer. Today, with the dominance of
> SaaS, that cost has been pushed back to the vendor. Most software companies
> pay big AWS or Azure bills every month – the more demanding the software,
> the higher the bill.

This irrational sheep mentality amuses me. Yes, tehre are some very specific
cases where AWS & ca. is clearly a better choice, but for the most cases I saw
the TCO with hosting it on premises or renting servers is much lower,
sometimes by an order of magnitude (in some cases even more). But people
insist on doing it because others do it. We'll soon have an entire generation
of engineers completely hooked on AWS & co. and not even realizing other
solutions are possible, not to mention lower TCO.

------
mtkd
AI on the algo side is only half the story -- it has to sit in a domain
specific framework to be most effective

I see a lot of 'bolt-on' tech emerging -- it looks mostly snake oil -- there
is no obvious way to be competitive against teams that baked it in to the bare
metal design

Also most commercial use-cases I've seen need effective ML more than anything
else

------
angry_octet
There are many problems which are simply impossible to do with traditional
optimisation or human analysis, that ML can do really well at. But I get the
sense that this is not the type of problem that these "AI" startups referred
to are addressing. Instead its like 'here is a problem I can charge for, with
some ML magic it will be easy'. This is classic snake oil.

Being able to sift/classify/analyse data with ML really can be a 'moat', an
extreme competitive advantage. But using "AI" doesn't automatically get you
there.

Separately, AWS is an expensive luxury, which is worth it if for some reason
you can't manage your own computers.

I really annoys me when analysts like this guy mangle together things which
are obvious and then comes up with an unsupported conclusion, like "second AI
winter is coming man".

------
blueyes
The A16Z piece makes all these points quite clearly. This editorial is trying
to put a finer point on a sharp knife.

------
pandascore
Agree mostly but he only talk about some AI start-ups that have a 1 to 1 model
or at best a 1 to few. There is some AI startups like ours which have a 1 to
many model. We use Computer Vision to collect data from video streams and sell
data and transformed data through our API. The output of our models is the
same for everyone.

Cost wise though it's clearly being not knowledgeable about how it works or at
least think all AI startups have huge training set. For many companies owning
your hardware for training is a very easy step to rationalise cost.

It feels like an article written about all AI companies but actually (very)
true only for some AI companies.

------
Zanneth
I wonder how much of the formidable amount of computing resources required for
deep learning can be attributed to wasteful and inefficient programming
practices. A lot of the ML libraries that I see are written in Python with
very little attention paid to aspects such as memory usage, cache coherency,
concurrency, etc.

If we focused on writing more efficient software instead of demanding bigger
and faster machines with more and more GPUs, would the cost of ML become more
practical? More importantly, as the author pointed out, would smaller
companies have a better chance at making advancements in the field?

------
magwa101
Here's what cloud gives you that is very costly to implement internally, cost
accountability. Analysts running the same queries over and over would peg
internal hardware all the time. When we went to the cloud, we made a budget
for each division, problem solved. Same with DS. Give them a blank check,
they'll spend it, manage to a budget, they'll do it.

------
amai
"(my personal bete-noir; the term “AI” when they mean “machine learning”)"

This is so right. Using a term "artificial intelligence" for machine learning
is like using "artificial horses" to describe cars. It is even worse, since we
cannot even define what "natural intelligence" actually is. Stop talking about
"artificial intelligence".

~~~
DonHopkins
Or "artificial swans" that "appear even more lifelike".

[https://www.louwmanmuseum.nl/ontdekken/ontdek-de-
collectie/b...](https://www.louwmanmuseum.nl/ontdekken/ontdek-de-
collectie/brooke-25-30-hp-swan-car-1910)

>The bodywork represents a swan gliding through water. The rear is decorated
with a lotus flower design finished in gold leaf, an ancient symbol for divine
wisdom. Apart from the normal lights, there are electric bulbs in the swan’s
eyes that glow eerily in the dark. The car has an exhaust-driven, eight-tone
Gabriel horn that can be operated by means of a keyboard at the back of the
car. A ship’s telegraph was used to issue commands to the driver. Brushes were
fitted to sweep off the elephant dung collected by the tyres. The swan’s beak
is linked to the engine’s cooling system and opens wide to allow the driver to
spray steam to clear a passage in the streets. Whitewash could be dumped onto
the road through a valve at the back of the car to make the swan appear even
more lifelike.

>The car caused panic and chaos in the streets on its first outing and the
police had to intervene.

------
etrk
I interviewed at some AI companies a year or two back. They all had teams of
people dedicated to support each client: to clean their data, train their
models, integrate the domain-specific requirements, customize UIs, etc. They
sold themselves as the next AI-powered mega-unicorns, but they were more like
boutique consultancies with no obvious path to scale up.

~~~
auxten
"Boutique Consultancy" is quite recapitulative for most AI companies for now.
But this may be the only way to empower their clients. One of these startups
will find the path to scale up eventually.

------
moandcompany
Related to the topic of marginal benefits of AI models versus their costs:

Green AI (Roy Schwartz, Jesse Dodge, Noah A. Smith, Oren Etzioni - 2019)

[https://arxiv.org/abs/1907.10597](https://arxiv.org/abs/1907.10597)

------
marmaduke
I sometimes contribute to methodology projects in neuroscience ("AI" for
scientists). The most tiring part of it is explaining essentially these things
over and over. Very interesting to see the sentiment vindicated in
Startupistan.

------
tzm
I view AI as the application of ML and ML as the implement (tool). Therefor,
tooling efficiency is a competitive advantage of good ML projects.

------
orasis
Nice article. The flip side of the coin is that all these “problems” are
potential moats for a well tuned ML company to use to defend market share.

------
atulkum
On the other hand some of the startup is doing absolutely fraud on the name of
AI.I went to a self checkout store (AIFI.io). I did not touch anything but
they charge me $35.10. According to the receipt I took 17 packs of snacks :)
These guys are doing fraud on the name of AI. They have no technology no
software just put up some camera and open a store so that they can defraud the
investor. Anyone can try if intersted [https://www.aifi.io/loop-case-
study](https://www.aifi.io/loop-case-study)

------
laktak
> “AI coming for your jobs” meme; AI actually stands for “Alien (or)
> Immigrant” in this context.

Finally a correct use of "AI".

------
MacsHeadroom
Well, duh. Unless you invent AGI you're always going to be fitting new models
for new clients. The best case scenario is getting bought by a client and
becoming their full-time ML tailor.

For a pure ML company to IPO they'd have to both solve intelligence and
manufacture their own hardware. FOMO screwed a lot of investors who would've
been better off buying Google stock.

------
bryanrasmussen
Generally the use of the phrase from a great height implies the height is one
of morality, intellect, or valor (each of these decreasing in usage), I'm not
exactly sure what the great height Andreessen-Horowitz craps from is composed
of - maybe money?

I think they may just be crapping on them from a reasonable vantage point.

~~~
KaiserPro
The height is not really about morals. Its more about the blast radius of the
shit.

~~~
darwingr
Or like “nuked from orbit”

------
NickKampe
I guess I won't mention Kubeflow here.....

------
rotrux
This is a terrific article. Two thumbs up.

------
allovernow
All of this might be true currently, but that's because this current first
generation "AI" (technically should just be called ML) is mostly bullshit. To
clarify, I don't mean anyone is lying or selling snake oil - what I mean by
bullshit is that the vast majority of these services are cooked up by software
developers without any background in mathematics, selling adtechy services in
domains like product recommendation and sentiment analysis. They are single
discipline applications accessable to devs without science backgrounds and do
not rely on substantial expertise from other fields. That makes them narrow in
technical scope and easy to rip off (hence no moat, lots of competition, and
human reliance and lack of actual software).

The next generation of Machine Learning is just emerging, and looks nothing
like this. Funds are being raised, patents are being filed, and everything is
in early stage development, so you probably haven't heard much yet - but these
ML startups are going after real problems in industry: cross disciplinary
applications leveraging the power of heuristic learning to make cross
disciplinary designs and decisions currently still limited to the human
domain.

I'm talking about the kind of heuristics which currently exist only as human
intuition expressed most compactly as concept graphs and, especially,
mathematical relationships - e.g. component design with stress and materials
constraints, geologic model building, treatment recommendation from a corpus
of patient data, etc. ML solutions for problems like these cannot be developed
without an intimate understanding of the problem domain. This is a
generalist's game. I predict that the most successful ML engineers of the next
decade will be those with hard STEM backgrounds, MS and PhD level, who have
transitioned to ML. [Un]Fortunately for us, the current buzzwordy types of ML
services give the rest of us a bad name, but looking at _these_ upcoming
applications the answers to the article tl;dr look different:

>Deep learning costs a lot in compute, for marginal payoffs

The payoffs here are far greater. Designs are in the pipeline which augment
industry roles - accelerate design by replacing finite methods with vastly
quicker ML for unprecedented iteration. Produce meaningful suggestions during
the development of 3D designs. Fetch related technical documents in real time
by scanning the progressive design as the engineer works, parsing and
probabilistically suggesting alternative paths to research progression. Think
Bonzi Buddy on steroids...this is a place for recurring software licenses, not
SaaS.

>Machine learning startups generally have no moat or meaningful special sauce

For solving specific, technical problems, neural network design requires a
certain degree of intuition with respect to the flow of information through
the network, which both optimizes and limits the kind of patterns that a given
net can learn. Thus designing NN for hard-industry applications is predicated
upon an intimate understanding of domain knowledge, and these highly
specialized neural nets become patentable secret sauces. That's half of the
most - the other comes from competition for the software developers with
first-hand experience in these fields, or a general enough math heavy
background to capture the relationships that are being distilled into nets.

>Machine learning startups are mostly services businesses, not software
businesses

Again only true because most current applications are NLP adtechy bullshit.
Imagine coding in an IDE powered by an AI (multiple interacting neural nets)
which guides the structure of your code at a high level and flags bugs as you
write. This, at a more practical level, is the type of software that will
eventually change every technical discipline, and you can sell licenses!

>Machine learning will be most productive inside large organizations that have
data and process inefficiencies

This next generation goes far past simply optimizing production lines or
counting missed pennies or extracting a couple extra percent of value from
analytics data. This style of applied ML operates at a deeper level of design
which will change everything.

~~~
scottlocklin
>The next generation of Machine Learning is just emerging, and looks nothing
like this. Funds are being raised, patents are being filed, and everything is
in early stage development, so you probably haven't heard much yet ...

Citations needed. Large claims: presumably you can name one example of this,
and hopefully it's not a company you work at.

I've seen projects on literally all the things you mention: materials science,
medical stuff, geology/prospecting -none of them worked well enough to build a
stand alone business around them. I do know the oil companies are using DL
ideas with some small successes, but this only makes sense for them, as
they've been working on inverse problems for decades. None of them buy canned
software/services: it's all done in house. Probably always will be, same as
their other imaging efforts.

~~~
allovernow
>Citations needed. Large claims: presumably you can name one example of this,
and hopefully it's not a company you work at.

Unfortunately this is all emerging just now and yes, I do work at such a
company, but I'm old enough to not be naively excited by some hot fad. There's
something profound just starting to happen but everyone is keeping the tech
rather secret because it isn't developed/differentiated enough yet to keep a
competitor from running off with an idea, yet. Disclosure is probably 1-3
years out of estimate.

>I do know the oil companies are using DL...as their other imaging efforts.

You're correct, and I happen to have experience in this domain - except there
are a handful of up and commers courting funds from global majors like Shell
and BP, and seismic inversion is near the end of the list of novel
applications. Peteoleum is ground zero for a potential revolution right now,
if we can come up with something before the U.S. administration clamps down on
fossil fuels.

But we're talking complex algorithms which consist of multiple interacting
neural networks. We are rapidly moving toward rudimentary reasoning systems
which represent conceptual information encoded in vectors. I'm jaded enough
that I wouldn't say we're developing AGI, but if the progressing ideas I'm
familiar with and Workin on personally pan out, they will be massive baby
steps towards something like AGI.

The space is evolving at least as rapidly as the academic side, which I think
is an unprecedented pace of development for a novel field of study. I can't
help but feel like these are the first steps towards some kind of singularity.
There's no question that we are on to something civilization changing with
neural networks, what remains to be seen is whether compute scaling will keep
up with the needs of this next generation ML. Even if research stopped today,
the modern ML zoo has exploded with architectures with fruitful applications
across domains. The future is here!

~~~
scottlocklin
I mean, the fact that you just wall of texted me with "trust me" doesn't
inspire a lot of confidence. You could at least point me at an impressive
paper or something!

I read all the NIPS stuff every other year; don't see anything game changing
in there!

------
lazzlazzlazz
Is the misspelling of "Andreessen-Horowitz" and use of "A19H" instead of
"a16z" intentional?

~~~
scottlocklin
I suck at spelling. If I was one of the cool kids I'd claim to be dyslexic.

~~~
yubozhao
hi OP. We built an open-source library called,
BentoML([https://github.com/bentoml/bentoml](https://github.com/bentoml/bentoml))
to make model inferencing/serving a lot easier for Data scientists in various
serving scenarios.

Love to hear your thoughts on our library

~~~
EamonnMR
I was really hoping that you where about to offer an ML framework to improve
spelling.

