
Why's that company so big? I could do that in a weekend - pyb
http://danluu.com/sounds-easy/
======
malux85
I often hear this logic from inexperienced engineers. My answer to them is:
Then why don't you do it? <grin>

Recently a junior engineer said this to me about Uber, and I helped him to
understand - His logic was "it's just a taxi app".

There were loads of Taxi apps before Uber and loads of them after Uber. Yet
none of them were uber?

Sure that a medium complexity CRUD app can do the pickup and scheduling, and
spit out some json to the app.

But what if you have 100,000 journeys taking place simultaneously, and you
must track their positions so that you can render the journey map on the
invoice?

Can you do live traffic planning and re-route around the busy traffic on the
fly?

Does your weekend include the full billing system? This is a mandatory
requirement for a business.

Can you spot busy periods and increase the fares according to the laws of
supply and demand? All online in realtime? Can you continuously A/B test this
to optimise profits?

Can you deal with the fact that maps and roads are always changing?

All of these and more are the things that make Uber, Uber, otherwise you're
just one of those shitty taxi apps that failed.

Still think you can build it in a weekend?

~~~
rfrey
I agree with everything you say, but that response, like the junior engineer,
focusses on the technology. So give people six months not a weekend -- there's
a good chance they still won't be Uber.

The success of many companies, and probably all of the unicorns, has nothing
to do with technology. The tech is necessary, of course, but so are desks and
an accounting department.

Internalizing that has been difficult for me as an engineer.

~~~
bunderbunder
At a previous job, we had serious development bandwidth issues. The company
was trying to solve that by hiring more developers, and all it was doing was
making things worse.

The root cause of the development bandwidth problem was that we wasted vast
amounts of time and energy on building misfeatures. And we built so many
misfeatures because we had way too much development capacity in relation to
our product planning capacity. So the BAs didn't have enough time to fully
develop ideas before sending them to engineering, and were instead stuck
throwing things over the wall when they were still half baked. All these half-
baked ideas took time to build, and then they took time to fix after they
bombed on the market, and they slowed everything else down since every new
feature is something you have to ensure still works when you make a change in
that part of the software. Day in, day out, we were taking on technical debt
to cover our design debt bills.

So then we tried Agile, and man oh man did that make things worse. 'Cuz now
instead of just lobbing things over the wall, the BAs had to be actively
involved in every sprint. There goes any opportunity to sit down and
concentrate, because now you've got a nattering horde of 20 developers all
trying to get clarification on last week's decisions, which leaves less time
to make this week's decisions, which means next sprint things will be even
more sketchy and ill-defined.

The correct solution would have been to hire more business analysts, and they
key thing we needed from those business analysts wasn't instructions on what
to build. It was instructions on what _not_ to build.

~~~
amag
> The company was trying to solve that by hiring more developers, and all it
> was doing was making things worse.

You don't mess with Brook's law!

~~~
knightofmars
I second this. Additionally, get a grip on your queues!

I have yet to find a better text on how to properly manage software projects
than, "The Principles of Product Development Flow: Second Generation Lean
Product Development"[0].

[0] [https://www.amazon.com/Principles-Product-Development-
Flow-G...](https://www.amazon.com/Principles-Product-Development-Flow-
Generation/dp/1935401009)

------
systemtest
I recently had a discussion about how hard it would be to have a system inside
a bus to announce the next stop. Sounds like a weekend project right?

Think a bit further:

\- Hardware needs to be resistant to harsh diesel engine vibrations

\- 3G/4G connectivity

\- Need a mobile data contract with local ISP

\- Software needs to be able to handle network disconnections

\- GPS needs to be able to pin-point at which stop the bus is currently
stopped, even with bad GPS coverage in larger cities

\- If the bus skips a stop because of a detour, the software should be able to
detect it and announce the next stop

\- The announcement should be bi-lingual for Airport busses

\- The announcement should work for people with hearing aids

\- Server needs to know all bus-stops

\- Server needs to know different stop types, such as bus terminals,
intersection stops, regular stops, hand-over stops, virtual stops

\- Server needs to be able both work of a yearly bus schedule and real-time
update

\- Server needs to output in an understandable JSON/XML because the government
subsidies demand an open-data format

\- Server needs to publish data to an open-data server because of the
subsidies

\- Because the bus-company is sponsored by European Union money the control
interface should be translated to German/French/English/Spanish

And now this weekend project takes a team of 10 engineers working for a year.

~~~
orf
All London buses do this. I don't think you need a server, 3G/4G connectivity
or even GPS really. You might be over thinking it.

~~~
masklinn
On the other hand, the overthinking in question (which is probably true for
bare-bones next stop indication) provides the base for my personal bus bug:
know if/when a bus is reaching a stop from outside the bus in question,
especially with geofencing, in order to know whether the bus I want to take is
running early/late/not.

~~~
orf
They actually do this as well (and many have live updating signs), so the
buses do have GPS linked to a server of some kind. It's normally absolutely
fine, but on the couple of occasions for me where it broke it seems that the
server doesn't conduct the bus in any way and kind of hopes the bus is on the
right route.

That's all speculation though.

~~~
masklinn
> They actually do this as well (and many have live updating signs)

I don't know about London, I've seen some systems where the stops had "live-
updating signs" but only provided the time of the next official passage. And
while I wasn't clear, my bug is not just stop signage but phone access as well
(that's most useful when e.g. it's raining cats and dogs and you want to avoid
waiting in the rain at uncovered bus stops)

~~~
ahussain
In London you can go to the TFL website and see live updating times for when
the bus will get to your stop: [https://tfl.gov.uk/travel-
information/stations-stops-and-pie...](https://tfl.gov.uk/travel-
information/stations-stops-and-piers/)

~~~
masklinn
OK, that's really cool. Definitely means they have "a server, 3G/4G
connectivity [and] GPS" though.

~~~
makomk
Not necessarily. I think some of the systems use radio links to a central
location and transponders at fixed points on the route to track the buses.

------
wellpast
This is such a straw man conversation. Of course it is a fallacy to think "I
could build that in weekend" and translate that to a claim about the
inefficiency of a company. I think that's what is being responded to in this
thread. But that's not the interesting debate.

A better consideration (imho) is this:

    
    
       For a given business/offering/product, what is a likely optimal number of engineers needed to continuously deliver the product's value efficiently? Then: How far is a given company from that optimal number?
    

This is a hard analysis to do. It is virtually impossible to quantify or put
this kind of evaluation in a laboratory. There are too many counterfactuals
that would be at play.

We really just have to consult our experience and consider which of these is
the more probable:

(1) A business/offering/product team with lots of money is really good at
utilizing its hundreds of staff. OR (2) It's not.

I've seen a few (2)s from the inside. I have never seen a (1). Therefore I
lean more probabilistically toward the belief that a company is needlessly
big.

What surprises me is that that's not where people on this thread are leaning
---but has anyone on this thread actually ever _seen_ a (1)??? If not, where
is this belief coming from? I've never seen a ghost, so I don't believe in
ghosts....

~~~
notacoward
Consider, for the sake of argument, the possibility that perfect efficiency is
_unachievable_. Almost physically impossible. As a company gets larger,
certain kinds of inefficiency in hiring and utilization become absolutely
inevitable. With that premise in mind, how many employees does a company
"need"? If it would need fifty in that universe of perfect inelastic spheres
with no friction or similar effects, how many times that does it _actually_
need in the real world of such efficiency-robbing factors?

In my experience, the answer is two _at best_ and more often, around five. The
incremental _work product_ from each new hire diminishes more rapidly than
many realize, but the incremental _revenue_ might still make such hires worth
it. As Dan points out, a few percent increase in efficiency from optimization
can be huge. It might take multiple people quite a while to achieve that, and
it will still be worth it. There might be another group of several people
trying something else that doesn't pan out, but that doesn't mean they were
unproductive dead weight either. Bigger companies have to take risks too. They
just take them differently, and the cost is more obviously attributed to them
instead of the market.

Maybe a _single_ startup could do what Twitter (for example) does, in fairly
short order. Even that's unlikely when issues of scale etc. are considered,
but that's not even the point. That single startup is not a valid point of
comparison, because it doesn't count the cost of failed experiments. Add up
_all_ of the startups trying to do what Twitter does, including the failures,
and that probably be a better estimate of how big Twitter (Uber, Netflix,
whatever) needs to be.

------
wellpast
There's a naïveté in saying "I could do that in a weekend" as a measure for
employee bloat, but that's making a straw man of the question.

It is nearly always more probable that a large company with several hundred or
more plus engineers doesn't need that many. That those engineers are there for
other reasons -- political reasons or speculating on new products, valuation
reasons, etc. -- than for the reason that those engineers are fundamental to
the top or bottom line of the company's core product/s.

Uber does not need 2000 engineers. That's a prima facie fact.

I've worked inside several large companies and saw this first hand. Hundreds
of engineers across teams doing "stuff", no doubt, but nothing that ultimately
or ever tied to the top or bottom line. The only ideas that I could come up
with for why these people were there were: perhaps the headcount helped
company valuation, perhaps we needed that many benchwarmers for when there was
company churn, maybe there is some political reasons a manager wants a
sizeable headcount on their team, or maybe it's just leadership ignorance as
to how things are getting delivered. But in no way could I see a concrete
justification for many teams and many engineers.

What am I missing?

~~~
notacoward
Maybe some of those engineers really were superfluous. Maybe some were
contributing to the bottom line in ways you don't understand or appreciate.
I've seen plenty of engineers take infrastructure for granted, discount the
value of research ("speculating on new products" is _important_ ), or even
dismiss support as a cost with no revenue upside. Most often, as those people
gain experience they also gain a broader view, allowing them to distinguish
between complementary roles and truly superfluous personnel.

------
eriknstr
I searched for
GTGACCTTGGGCAAGTTACTTAACCTCTCTGTGCCTCAGTTTCCTCATCTGTAAAATGGGGATAATA and found
another post by the same author on another website
[http://bitfunnel.org/strangeloop/](http://bitfunnel.org/strangeloop/)

I've seen some other posts by Dan before but didn't know what kind of things
he works on because I generally don't check that because if I did then I'd end
up just checking people instead of reading what people were writing. Anyhow,
from the article linked above -- for which a video is available at
[https://www.youtube.com/watch?v=80LKF2qph6I](https://www.youtube.com/watch?v=80LKF2qph6I)
\-- it turns out that Dan has done quite a bit of work with implementing
search related technology, and I think that was interesting. His contributions
to BitFunnel can be seen at
[https://github.com/BitFunnel/BitFunnel/graphs/contributors](https://github.com/BitFunnel/BitFunnel/graphs/contributors)

~~~
alva
I thought you had accidentally pasted some of your genetic code from your
clipboard. _future internet problems_

------
YZF
The other side of the coin is that big companies do move slow.

There is a lot of communication overheads, a lot of bureaucracy, backwards and
cross compatibility concerns, conservatism, wasted duplicate effort, wasted
pointless effort, some people who aren't contributing much etc. etc.

This is why tiny startups can sometimes dethrone a large leader. However
that's often a multi-man-year effort.

So I'd rephrase the statement: "Why's that company so big? I could do that
with 20 brilliant engineers and 18 months". I think that order of magnitude
can attack pieces of any established software business from a technical
perspective. Then there's obviously the business side.

~~~
bastijn
That would get you your first launch product maybe. Not an actual enterprise
product. Sure, people forgive the small startup for their crashes, their
unfinished and unpolished app, their weak customer support, their performance
hiccups, etc. As long as they bring something that solves an actual new
problem that isn't solved by another product easily available. But an
enterprise product, no way you will make that in 18 months with your 20 people
while keeping up the maintenance burden you introduced in v1 when you took
some shortcuts.

------
yomly
Yup. Have never gotten the whole "Airbnb/Uber/<startup>" is just a crud app
line of thought.

If they're crud apps then everything else in between is just glue programming.

The author of the post covered some very astute points but one not covered in
much detail is the challenge of scale and reliability. Maybe this falls under
the bucket of optimization and/or latency but I think this deserves being
called out on its own.

Once you have Uber-scale number of users requesting taxis at any point, and
have a network of drivers constantly communicating their position with the
app, suddenly you no longer have a trivial crud app on your hands...

~~~
new299
These companies are not "tech" companies in my book any more than the Sears
catalog was a "tech" innovation.

They are normal business that happen to be enabled by technology, but the
technological challenges are not new or to be frank uncommonly challenging.

Because we're I assume mostly coders, we think they are differentiated by
tech, almost everything else (marketing, sales, customer service) is likely
more important.

Bad technology could certainly kill a platform, but excellent tech won't save
it, and a mediocre implementation is good enough.

~~~
Cyph0n
If you think scaling a service like Uber is not challenging, you're vastly
underestimating how large they are.

Uber especially has requirements that make it tough to scale correctly. You
need to track each Uber session in near real-time, right? That means you
simply cannot afford to drop or interrupt those sessions. It's a completely
different problem than scaling Instagram or WhatsApp.

~~~
new299
Hmmm... nothing like near real-time. Challenging near real-time is sub-
nanosecond.

Uber real-time requirement of what < 30s? (for a viable product).

No that doesn't seem like a challenging problem. It seems like a problem that
would require almost no throught for a MVP, and then incremental challenges as
the problem scales.

It seems like a problem that is so comparatively easy compared to what
computers are capable of that I wouldn't even class it as "difficult".

But what's telling about your answer is that your thinking about this
technical challenge, this feature as being of primary importance.

Do you think if Uber was 10 times cheaper than a regular taxi real-time
tracking would be as significant?

Uber's funding which has enabled them to subsidize their expansion (and
rides), advertising and promotion and possibly that they plugged in to a
easily accessed communications channel (mobile apps). Were likely all more
important than the quality of their implementation.

~~~
Cyph0n
Probably less than 30s between pings during a ride, but I haven't tested it
tbh. I usually use the word real-time when talking about systems that operate
in the ns time range, and near real-time when talking about software. Sorry
for the confusion.

What I was trying to emphasize is that Uber needs to keep track of these
sessions and make sure they're not interrupted, or else they lose customers.

Yes, I know building something that fits these requirements and works _most_
of the time is trivial. But we're talking about a system that must basically
work as close to always as possible. More importantly, it must do this at
scale, maintaining and keeping track of perhaps millions of sessions at once.

In a nutshell, an Uber MVP is probably not difficult, but the transition from
MVP to real-world product is non-trivial in my opinion.

I'm actually planning on setting up an Uber-like service for my home country,
which is why I've been giving these issues some thought.

~~~
problems
Even 1M simultaneous users reporting every ~30 seconds could average down to
~30k messages per second. And they'd be extremely trivial to segment between
servers. You're right, not a weekend hack, but a few weeks of a small team's
time isn't out of the question. The technical side is not the challenge here
though really - the business side is.

~~~
Cyph0n
Perhaps you're right when it comes to scaling, but you're assuming that the
system was designed correctly in the first place, which would be impressive
imo. There is also a lot of failure handling and recovery that needs to be
done gracefully, which I think would be complicated for a session-based
system.

Again, I'm no expert, just making educated guesses!

------
bsandert
To me this is a form of the Dunning-Kruger effect[0], where only somewhat
informed people estimate the cost of a "0.1" (or POC) release without
considering the famous "unknown unknowns" which may involve scaling, billing,
3rd party integrations, etc.

[0]:
[https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect](https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect)

~~~
vidarh
I find I've had to bar myself from thinking about many of these factors, after
I realised how many projects I stop myself from pursuing because I know about
the hard bits, and realise how many of my past projects I'd never have taken a
shot at if I knew what was coming up and was thinking like that.

It's great to know, but often it's not so great if it leads to inaction...

E.g. if you have a "I could do that in a weekend" impulse, sometimes it's
worth following that impulse. If you're right, you've spent a weekend building
something potentially great. If you're wrong, you've spent a weekend learning
about things you clearly didn't know in advance, and the lessons can often be
very different from the objections you'd have throw up if you thought about it
without doing, and sometimes what you learn may provide the impulses for
another new idea.

But it's _hard_. E.g. my first impulse for anything that requires collecting
money is how painful it is to deal with credit card processing and reporting
and VAT and accounting, because I've done big complex billing systems with
heavy reporting requirements, and compliance complexity etc.. It's very hard
to put that impulse aside, and "just do it" and think about how to solve the
payments afterwards. But that's how I did things he first few times I needed
to handle payments.

------
lmm
And yet sometimes a small team really could do it. I spent years complaining
about MSN/AOL/Skype etc. and talking about how a small team could build a
better messenger in a week, but we didn't really believe it. Then Slack
actually took that possibility seriously, and look at the results.

~~~
cocoflunchy
Sure, they were a small team in the beginning. Google too... Now they probably
have more than 500 employees. (They had 385 in february 2016:
[http://uk.businessinsider.com/slack-ceo-worried-about-
growin...](http://uk.businessinsider.com/slack-ceo-worried-about-growing-
culture-2016-2))

~~~
bluedino
Slack also jsut recently hit 3 million users. 100 million people were using
AOL Messenger at it's peak, and 12 million were using ICQ when AOL bought
Mirabillas.

------
merb
Actually it actually wrong to assume somebody could solve a trivial problem in
a weekend, however some companies still have way too many people working at
it. Consider twitter with their 3860 people. they could probably do it with
half of them. especially the technical people are 40% of the company... way
too much. i don't think that they need no people but well...

~~~
r_smart
Yeah, this post seems to be addressing an annoying conversational experience,
but ignores the other side: the reality, which you describe. I understand that
just scaling out their platform is a ton of work that justifies having a lot
of engineers. But how does Twitter have so many engineers on staff and yet
their users constantly complain that they don't improve even seemingly low
hanging fruit? Is ~1,500 people not enough? Is there some lingering technical
debt they're dealing with? Or is it just people accumulated over time they
value too much to let go?

------
FussyZeus
I personally have never seen this complaint leveled against the mighty Goog,
nor really any company that's cash positive. When I see people giggle at
bloat, it's more often than not twitter and still more often than not, it's
not intrinsically because big company = waste, but because company that should
be small is made to be big either by VCs or Wall Street, and then can't turn a
profit anymore.

No, maybe if twitter was still a tiny company you couldn't use it in Arabic
but maybe they also wouldn't constantly be on the edge of going out of
business.

Edit: I think this is also a regular criticism of Twitter because Twitter has
spent the last several years trying to Facebook itself and strayed far from
the original idea. You can argue the pros and cons all day since they're
pretty much infinite on both ends, but Twitter as a company has spent vast
amounts of money on a lot of features that none of it's original userbase
really wanted (and drove a lot of people off in the process).

Me, I only really ever used it as a time-waster so I'm not awfully attached
but I listen to a ton of podcasts and I can't count the number of hosts who
disliked more or less every new thing Twitter did.

~~~
tim333
Things like Dropbox are striking. I guess it was mostly written originally by
Drew Houston over a year or so and was cool.

Then 2011, 50 million users 70 employees. Seems fair enough.

Now about 1500 employees. It worked fine and scaled fine with 70 employees so
you wonder if hiring the other 1430 was good business. I guess if VCs are
throwing money at you you may as well spend it on something. Though that's
probably what Yahoo was thinking when it went to 11,700 employees and that
didn't play out so well.

~~~
nazgob
I bet its mostly sales. I had no idea about how much sales ppl you need at
certain scale.

~~~
devonkim
Sales persons are necessary when continuing growth requires efforts that do
not scale as effectively. Marketing and advertising campaigns work
sufficiently well for b2c with 1-2 people but you hit a wall if you want to
selling to, for example, the Pentagon because the way those with the deepest
pockets buy is nothing like how most efficient individuals and organizations
buy. Groupon's massive needs for salespeople is an example of how even lower
margin b2b even needs sales due to how scaling out b2b is so much less cost-
effective. But for investors, landing a huge contract is not as much about the
revenue as much as an indicator of stability in terms of both revenue and
market signal (no Fortune 100 will buy some random fly-by-night vendor's stuff
without code escrow even if it's the best thing ever because their sourcing
overhead is so poor and big companies are immensely risk-averse by design).

~~~
tmnvix
Atlassian's 'b2b without salespeople' approach has worked well for them.

~~~
devonkim
They definitely have sales roles in more recent years and have a sales ops
team [1][2][3]. The Federal channel manager description even acknowledges that
there's unique challenges in selling to the government that necessitates
someone that knows the different contracting vehicles. What they've done with
little turn to sales culture is pretty admirable but at the same time only
possible with organic, engineer-to-engineer driven growth Github and Atlassian
suites.

Salesforce grew somewhat similarly via rogue accounts on expense accounts, but
to actually clinch the biggest growth there was no way to avoid salespeople to
get over $500MM in revenue probably.

[1]
[https://www.smartrecruiters.com/Atlassian/99086565](https://www.smartrecruiters.com/Atlassian/99086565)
[2]
[https://www.smartrecruiters.com/Atlassian/95395260](https://www.smartrecruiters.com/Atlassian/95395260)
[3]
[https://www.smartrecruiters.com/Atlassian/93068238](https://www.smartrecruiters.com/Atlassian/93068238)

------
new299
The sad truth is that in the majority of cases the quality or time taken over
the implementation isn't particularly important other factors matter much
more:

* building something people actually want. * advertising * engaging with users * aggressively promoting your product. * customer service

It doesn't matter if what you did was hard or not. It matters that you can
acquire and retain customers.

------
erichocean
> _Businesses should keep adding engineers to work on optimization until the
> cost of adding an engineering equals the revenue gain plus the cost savings
> at the margin. This is often many more engineers than people realize._

I think this is the key realization, and it's actually a variation on Conway's
Law[0]. In a nutshell, some part of BigCorp discovers (or believes) that by
hiring a software developer for $X, they can make $Y > $X in return. So they
do.

As long as some part of BigCorp is able to do that, you get more software
engineers. This continues until you either (a) run out of budget, or (b) run
out of opportunities.

Because it's not done "top down", it less efficient than it could be to get
all of the features in total. But due to _how_ these engineers are funded
internally, it's not possible to do the "top down" approach at all. And that's
okay, all that really matters is that $Y > $X.

[0]
[https://en.wikipedia.org/wiki/Conway%27s_law](https://en.wikipedia.org/wiki/Conway%27s_law)
"Organizations which design systems ... are constrained to produce designs
which are copies of the communication structures of these organizations." In
this case, it's not the communication structure that causes the perceived
bloat, it's the funding mechanism. The basic idea that software organizations
reflect some non-software, human structure in the business, holds.

------
Animats
As of 2008, the core search team at Google was just under 100 people.

Then came search customization. Suddenly, no more cached replies.

~~~
draw_down
100 people?! I could do that myself, in a weekend.

------
notacoward
Everything seems easier when you're not doing it yourself. It's only a minor
variation on the grass being greener. Everything gets way more complicated
when you're doing it at scale, with high reliability, for real users, with
some hope of making money. Practically nobody adds complexity for its own
sake. Usually, if you think something is much more complicated than it needs
to be, it's because you don't have any idea what the real requirements and
constraints are.

------
dools
To me this post addresses the same naivety that David Graeber demonstrates
when he claims that everyone not involved in hammering out a sheet of steel to
make a horseshoe or pulling a plough is engaged in a bullshit job[1].

I think it's not a coincidence, therefore that many of the people I run into
who advocate a universal basic income, arguing tooth and nail against a job
guarantee (the YC guys no different) is young, tech savvy and overly
optimistic about the scope of automation to subsume all human labour within
months.

[1] [http://strikemag.org/bullshit-jobs/](http://strikemag.org/bullshit-jobs/)

~~~
Houshalter
Graeber's biggest point is that even when he talks to people and asks them if
their job is valuable, and _they should know better than anyone else_ , they
often say 'no, not really.' I don't think he thinks "anyone not involved in
hammering steel" is unnecessary, that's a bit of a strawman. The argument goes
that big corporations have a tendency to hire more people than necessary
because middle managers are rewarded based on how many people work under them.
Or that people looking to keep their jobs, try to make themselves look more
busy and valuable than they really are.

People waste a lot of time at offices doing things that aren't work. But our
culture demands we be at work 8 hours a day, even if the work only requires 4
hours. Economists in the 1930's thought the world would be so rich by today,
that people would only need to work a few hours a week. And the economy has
grown vastly since then, and we are much richer. But I don't think most people
have the option of working that little, even if they want to.

But that's a different subject entirely from automation. Those jobs can really
be valuable and still be vulnerable to automation. AI has improved a lot in
the last 5 years. I don't think it's implausible at all to imagine most jobs
done now being partially or totally automated.

~~~
dragonwriter
> Graeber's biggest point is that even when he talks to people and asks them
> if their job is valuable, and _they should know better than anyone else_ ,
> they often say 'no, not really.'

I don't think that's really true, especially in a big corporation.
Transparency of the value pipeline in such institutions isn't great at any
level, and big corps (and similar large institution -- government often has
the same problem, for instance) have a tendency toward _not_ making even the
concept of the value stream that justifies the position known to the people
working in it much of the time. Its quite likely that the people working a
position don't understand how its _supposed_ to deliver value (and are in a
poor position to see the big picture of whether it does), while the people who
do know how it is supposed to deliver value don't see the small picture of
what actually goes on (and thus are in a poor position to evaluate whether it
_does_ deliver the value it is supposed to.)

There's no particular reason to think the people working in the position are
in the _best_ position to evaluate value delivery (or that even the people
_in_ the best position, whoever they might be, are better at answering the
question than a Magic 8-ball.)

~~~
sedachv
There is a great incentive to obfuscate when your "value delivery" is skimming
(mutual funds that charge 1.25% management fees for underperforming the S&P),
privatization fraud (health insurance in the US, government contractors, and
pretty much everything related to real estate since "home appreciation" is now
supposed to pay for people's retirements instead of pensions), taking
advantage of stupid people (lottery, penny auctions, telemarketing, almost all
advertising), education fraud (textbook industry, "student loans now, regret
later" private colleges and universities - which since many student loans are
federally guaranteed is also a form of privatization fraud), or "intellectual
property" racketeering.

------
ErikAugust
Same thing happens in software development at the library level:

"Why not use [Example OSS Library]?"

"Nah, I can do it in 15 minutes."

"You know, they have [X] contributors, lots of great documentation, and 6
months of work into it."

"Yeah, I know [but whatever.]"

~~~
jkchu
I get the point that you are making, but if something _really_ only takes 15
minutes to do, I wouldn't necessarily use an open source solution just because
one exists.

Depending on the situation, the library may have a ton of extra bloat that
your application will never use. Also, recklessly deferring to open source
libraries can potentially snowball into something like the left-pad fiasco[1].

[1]: [http://blog.npmjs.org/post/141577284765/kik-left-pad-and-
npm](http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm)

~~~
ErikAugust
Yeah, and I get your point too. Eight lines of code? Yeah, you don't need a
library. But usually when someone says "15 minutes" they mean 1 - 2 days. So
things of that complexity is when it's obviously you should choose a good OSS
lib.

~~~
hex13
It depends. In many cases you would be right, in many other cases you
wouldn't.

It depends on many factors:

\- what is the problem? (some problems are really hard, some problems are
trivial and just seem hard)

\- is developer experienced enough to build own solution?

\- are existing solutions good or badly written?(I often refrained from using
existing solution when I looked on bloated code in it).

\- is solution suitable to project? ("too much bundle size", "doesn't support
X", "not too configurable", "too slow", "too hard to use").

etc.

------
azdavis
Completely unrelated to the article, but I'm curious what people's thoughts
are on full-width lines of text characteristic of unstyled HTML.

I've seen some[1] bemoan its impacts on readability and others[2] complain
about websites which artificially restrict themselves to (what can be) a
narrow slice of the viewport.

[1]:
[http://bettermotherfuckingwebsite.com](http://bettermotherfuckingwebsite.com)

[2]:
[https://plus.google.com/+LinusTorvalds/posts/2Ndtw3ZBRX2](https://plus.google.com/+LinusTorvalds/posts/2Ndtw3ZBRX2)

~~~
rch
On desktop I reflexively zoomed in to %125 and didn't think about it until I
read your comment.

It also has a reasonable default font size my phone, so it's easy to read
quickly. This is pretty rare since browsers gave up on text reflow.

~~~
hex13
I begin to think that nowadays websites are just designed for "zooming in" in
mind (very small fonts, too many space etc.).

So maybe we shouldn't complain. These websites are just meant to be zoomed.

Although some web developers are cruel and block zoom with meta tag (I never
understood why some developers did that, maybe for trolling people with poor
sight?).

------
VestingAxis
>>There’s also a wide body of research that’s found that decreasing latency
has a roughly linear effect on revenue over a pretty wide range of latencies
and businesses.

Anyone have pointers to such research?

~~~
jedberg
There is no published research I'm aware of. However, you can find an article
about every major tech company talking about how reducing their latency
increased whatever their core metric was (not necessarily linearly though,
sometimes it was and even steeper curve). I know for sure this is true at
eBay, Netflix, and reddit because I've seen the numbers personally, and I have
friends at Google, Amazon and Facebook who have said the same.

------
informatimago
Otherwise, indeed, it's a basic evaluation error, as described as long ago as
in the The Mythical Man-Month; it just seems that nowadays, the multiplier is
not 9, but even bigger.

[https://archive.org/details/mythicalmanmonth00fred](https://archive.org/details/mythicalmanmonth00fred)
Figure 1.1, page 5

To go from a program to a programming system (interfaces, system integration),
you need to multiply the effort by 3.

To go from a program to a programming product (generalization, testing,
documentation, maintainance), you need to multiply the effort by 3.

So to get a programming system product, you need to multiply the effort from
the program step, by 9.

Nowadays, we need to compose indeed multipliers for distribution, reliability,
availability, localization, targetting multiple UI platforms, etc.

And the original multipliers are also probably too small, when you see the
effort required just to integrate testing and CI on some projects.

------
FLUX-YOU
I have worked with one-man companies that did this and often they are shoddy
products. Healthcare just seems to suck at software.

Full-stack developers are already a very fast evolution from the front/back
end developers, which were an evolution from a guy doing just HTML/CSS in the
90s. Now you have to include sales, marketing, negotiation, project
management, and business management skills on top of your full-stack
development skills. The result is that something is going to suffer unless you
are a genius, the product is super-simple, or you have 20-30 years to have
learned all of this.

There's a much more interesting question in the form of: "Why's that company
so big? A tight-knit team could do that in a weekend".

------
handedness
The technology of a big company is often a necessary condition, but it's
seldom (if ever) a sufficient one.

We technically-minded people are often pretty terrible at accurately picturing
reality as it exists outside of our respective spheres. Consider:

"Why is Armani a billion dollar company? I could make a beautiful suit in a
weekend," said the talented seamster/seamstress.

------
dantiberian
> Recently, I was curious why an org that’s notorious for producing unreliable
> services produces so many unreliable services. When I asked around about
> why, I found that that upper management were afraid of sending out any sort
> of positive message about reliability because they were afraid that people
> would use that as an excuse to slip schedules. Upper management changed
> their message to include reliability about a year ago, but if you talk to
> individual contributors, they still believe that the message is that
> features are the #1 priority and slowing down on features to make things
> more reliable is bad for your career (and based on who’s getting promoted
> the individual contributors appear to be right). Maybe in another year, the
> org will have really gotten the message through to the people who hand out
> promotions, and in another couple of years, enough software will have been
> written with reliability in mind that they’ll actually have reliable
> services. Maybe.

This sounds like it could be Apple?

------
shubhamjain
Besides underestimating the scope of the problem, one logical fallacy that
programmers fall for is believing that a single developer should be able to
juggle lots of tasks easily. If a small team can create a SaaS application,
why do large companies need so many engineers for so many specific things?

I think, at a certain scale, it becomes necessary for organisations to ensure
that there are certain people keeping specific things running. Let's say, a
company doesn't have any specific person assigned for server administration
and then one day, is confronted with a DDoS attack. Who is supposed to handle
it? Whose responsibility is it? You have few people who know the basics of
operating servers but no one who can swiftly respond to it. If no clear role
is established, then the company is bound to face more attacks and possibly,
lose more customers.

------
buckbova
Looks like my primary search engine DuckDuckGo tackled this with a relatively
small team.

[https://en.wikipedia.org/wiki/DuckDuckGo](https://en.wikipedia.org/wiki/DuckDuckGo)

[https://duckduckgo.com/](https://duckduckgo.com/)

~~~
linkregister
At an Alexa rank of 676, that's pretty impressive. I think the key to DDG's
success with a small set of engineers is their small scope.

If they ever wanted to expand their product scope, for example, index their
own searches instead of aggregation, they would start running into the
obstacles discussed in the article. The Lucene comparison in the article
doesn't apply to DDG (yet).

------
fencepost
I was on the other side of this with a product that I was one of the main
people on back in the late '90s - I looked at what we were selling and how
much we were selling it for, and thought "Why are people buying this? Anyone
with a decent IT department should be able to do this in-house."

But really when you sat down and looked at it, what we were selling was a
supported and tested product that could be configured and active in less time,
with excellent ROI, and at significantly less than the cost of dedicating an
in-house developer to do it. What we were selling wasn't cutting-edge
technology, it was time and cost savings for a department that was almost
certainly already short of both money and staff time.

------
cloudhead
I think a lot of people are willing to "leave cash on the table" if it means a
smaller, more effective team and leaner product.

~~~
Yokohiii
Certainly not if you are VC driven.

------
sliken
I'm amused the central topic of this post is that google indexes 50M primes in
random files in github, which they seem to not. 961748729 is one of the primes
just before the 50M'th

Your search - site:github.com 961748729 - did not match any documents.

------
Lxr
The article raises some good points but there is still a dichotomy here to me.
Compare a successful startup with 2 founders with an established company with
2000 employees in the same business. The scale of the difference in employees
vs the difference in the product is often shocking. BigCo product might be
better, but that much better?

Obviously the quality of the product won't scale linearly with the number of
engineers, but still, startups punch above their weight. Maybe it just
highlights how much harder things get when the company reaches a certain size.

~~~
tmnvix
Those 1998 extra staff don't have to make the product 1000x or even 2x better
(however you might quantify that). They just have to make it sufficiently
better to maintain or gain market share.

------
NKCSS
Weird example, where it's assumed that a single machine can only hold 50GB of
document data, which results in the assumption that you'd need 200k machines
to store 1T documents...

------
payne92
Apps aren't just a pile of features, they're also ecosystems.

In the early years of Twitter, it would have been relatively easy to clone
Twitter, but you won't have had the ecosystem.

Poster example: Google+

------
willvarfar
As a counter point, on every product I've worked on, on every time we've
wanted to make a 'big leap' we've need to get a small expert team to go make
it happen.

~~~
xixixao
Dod that small expert team polish the product and maintained it for the next 2
years?

Something about 80%/20% rule...

------
ThomPete
I call this the difference between building a product and a business.

It's relatively easy to build a product and even to ship it.

The real art in building a business to support it.

------
mahranch
> we want to maintain an artisanal search index of 1B documents. Then our cost
> comes down to $12M/yr.

Yeah, no. It doesn't cost that much to maintain an index of that size. Not
even remotely close. My roommate coded a search engine back in 2001-02 as part
of a class project/hobby and it easily had an index that large, probably
larger till he shut it down. The key lies in not crawling it all in a single
day. And your index doesn't populate overnight, it takes months of slow
crawling. You can maintain an index of practically unlimited size for chump
change (well, compared to the millions OP was throwing around). Do people
think Excite, Hotbot, Lycos and Yahoo! in the early years had $12 million per
year to spend on their indexing? Hell no. Every single one of them would have
went bankrupt in a week. Those companies weren't even worth that much back in
the mid to late 90s. Your biggest cost is going to be user loads on your
servers (CPU/Ram/Bandwidth), not maintaining an index.

------
bdrool
I doubt most who say this sort of thing mean it literally (although no doubt
some naive sorts do, but they usually are self-identifying through the other
outrageous things they say, and thus can be ignored). But one counterpoint to
keep in mind is that the alternative view is every company is running as
efficiently as possible. I doubt anyone really always believes that, either.
For example, when a story about Twitter was posted on HN recently, it was
widely discussed that they probably don't need anywhere near as many engineers
as they actually have, and could perhaps reduce costs to the point where they
are at or near profitability just by reducing staff. There might be some truth
to that. Point being, it's not that simple, and there are no absolutes.

------
programminggeek
Building an app and building a business aren't the same.

Go out and sell the app to people and find out how easy it is.

------
Hyperized
Always reminds me of: [http://i1.wp.com/beckybendylegs.com/wp-
content/uploads/2014/...](http://i1.wp.com/beckybendylegs.com/wp-
content/uploads/2014/01/Modern-Art.jpg)

------
infinite8s
From the article:

> There’s also a wide body of research that’s found that decreasing latency
> has a roughly linear effect on revenue over a pretty wide range of latencies
> and businesses.

I'd be interested in seeing that citation list, particularly for non-
google/amazon businesses.

------
thallukrish
I feel if you have something better to offer, say a better way to look at
information, not just a better page rank algo, but a different way to look at
information that is scattered across, one that enables a better insight into
existing web pages and less overhead of constant 'searching' by typing a
query, something that automatically helps to "see" relevant information with
less effort, then it can be an alternative to what a Google search does today
and can position itself as a competitor in the long run even if it does not
exist today.

------
nickbauman
If your problem is small, viable solutions to that problem are vast. If your
problem is large (scale), there's usually only one or close to only one way to
solve it. So choosing the right path is important. Since you can't predict
product market fit (you cannot predict value), you have to figure out how to
iterate and test as close to continuously as possible. This last thing is what
most people just can't do. Programmers included. So you can see there's a
technology side and a culture side and they have to be well-aligned.

------
WalterBright
Even if you're just building an open source app to give away, sure, you can
get 99% of it done in a weekend. The last 1% will then consume 99% of your
time (if you want anyone to use that app).

------
nlawalker
From 2009, still relevant: [https://blog.codinghorror.com/code-its-
trivial/](https://blog.codinghorror.com/code-its-trivial/)

------
itsyogesh
The question I was wondering about is: When does an organization realize and
say, we have hired enough engineers to support our current systems?

------
chias
Jeff Atwood made a fantastic writeup of a very similar sentiment about
StackOverflow / the StackExchange network:
[https://blog.codinghorror.com/code-its-
trivial/](https://blog.codinghorror.com/code-its-trivial/)

Until reading that article, I often espoused similar views on applications. It
was a really eye-opening read.

~~~
dlubarov
Fog Creek is something like 50 employees right? That seems like a reasonably
sized team for their product, but Twitter/Uber are in a different ballpark.

------
eonw
I agree and disagree, I've been with companies and seen how scope can expand
as you grow, building internal apps, monitoring, reporting, all kinds of
stuff. On the flip side i know a guy that managed a team of 5, the 6 of them
total managed the snippet editor in Visual Studio. Six people to manage a
snippet editor? Really?

------
brachi
Reminds me of this great article about the 'this shit is easy' syndrome:
[http://steve-yegge.blogspot.com.ar/2009/04/have-you-ever-
leg...](http://steve-yegge.blogspot.com.ar/2009/04/have-you-ever-legalized-
marijuana.html)

------
HugoDaniel
Hmm somehow it reminded me of brook's law CEO:

"Give me 9 women and I'll give you a baby in 1 month"

------
TazeTSchnitzel
Features and scale are huge things.

Maybe you could make, say, a Twitter clone, in a weekend. But you couldn't
make one that can handle anything near Twitter's scale nor that is as feature-
rich as the actual thing, and besides, nobody will be using it.

------
matchagaucho
To this I draw a 2x2 matrix on a whitboard. In each of the 4 quadrants I
write:

"Product and Engineering"

"Ops"

"Sales and Marketing"

"Legal"

Enter tasks and goals for each quadrant. Iterate.

This is the minimum effort required to sustain any app. Developers are mostly
only aware of the first quadrant.

------
candeira
One of my favourite tech jokes used to be "Google? I could build that out of
wget, grep and baling wire!". The fact that "I could build that out of Lucene
and money!" is not intended as a joke gives me pause.

------
logicallee
Anyone find the 528 MB C++ file that is referred to in the Twitter screenshot
near the end of the article? (With only 57 lines of code but, oh, an array
with the first 50M primes hardcoded.) What's that about?

~~~
omaranto
I'd guess it is probably for Project Euler.

~~~
logicallee
that "probably" makes me wonder why you would say that? is there something
about 50 million primes in there?

~~~
omaranto
I don't think you really need that many primes for Project Euler, but I can
certainly imagine someone tired of including a call to her prime sieve in tons
of problems, and of having to waiting for it to run, and thinking, "let's just
make a library that has all the primes I could possibly need in a constant
array".

------
GoToRO
"I could do that in a weekend"

You could but you didn't so I can't use it...

------
kriro
Sometimes it takes a lot of time to reach something that is relatively trivial
once you know how to do it.

I can reproduce E=mc² in a matter of minutes. Doesn't mean I could have ever
come up with it in the first place.

------
spectrum1234
What exactly is the point here? At scale, it's not the tech that got the
product to the finish line.

The endurance part of the company has nothing to do with tech or even scaling
or optimizing that tech. Not even close.

------
informatimago
In France, the law (Art. R. 4228-10 du Code du travail) imposes one WC and one
urinal per 20 male workers, and two WC per 20 female workers. So basically, it
expects that up to 1 worker per each 10 workers will be busy with non-job
related stuff.

Another datapoint, I heard that an accountant counted that the actual number
was 1 for 50.

In any case, when you're alone or a small startup, this doesn't seem much, but
when you have 100,000 employees, this means that between 2,000 and 10,000
equivalent employees are all the time in the WC (and being paid for that,
basically).

Of course, there are a lot of other overheard that become very significant
when you have a big corporation, but that still exist (and that we easily
discount) for small startup and individuals.

~~~
cauterized
Ever thought that maybe that 1:20 or 1:10 isn't about handling steady load but
about handling pre-/post-lunch peak loads without half hour lines?

(And I've seen the half hour lines, in an office that had two WCs for 100
people. THEN you end up with people spending 10% of their day doing something
other than work.)

------
olalonde
That $12M/yr figure for indexing 1B documents seems way off to me.

~~~
astrieanna
Too high or too low?

~~~
olalonde
Too high. See [http://news.appbase.io/scaling-elasticsearch-
writes/](http://news.appbase.io/scaling-elasticsearch-writes/) for example:

> In doing these tests, we indexed over 1 billion documents under just two
> hours. The peak merge rate we saw was 129 MB/s, yes our ElasticSearch
> cluster on a 24 node setup is gobbling data at this speed.

> Most mind boggling of all: At our peak sustained ingestion rate, we would
> have added over 13 billion documents in a single day and this would have
> cost us a mere USD 244.

Their test documents are much smaller (100 bytes) than a typical web page but
if you multiply that figure by 50 to get to 5kb documents (probably a
realistic average web page size minus all markup and assets), you get 12200$
for 13 billion documents or 1220$ for 1.3 billion documents (assuming that
scales linearly). I'm probably way off in my estimate myself but I'm convinced
that a few thousand dollars is much closer than the proposed 12M$.

------
Xyik
Glad to see some sanity for once and people that realize building 'trivial'
apps like Twitter and Uber are not feasible 'in a weekend'. Just making sure
the app is secure takes months, and security becomes more complex as features
are added. Pretty much every feature you add to the app can explode the
complexity exponentially.

e.g: a sorted timeline if strings is easy, but add the simple feature of
'retweeting' and being able to 'unretweet' for example and things quickly
become more complex.

------
b34r
Those numbers are whacked. We use ElasticSearch with >10B docs ingested and
the cost is like a few thousand dollars a month for hosting.

------
interdrift
Yeah right, except that you can't. It takes so much time to nurture something
and build it in a way that satisfies the user.

------
kristianp
I really enjoy how this guy's blog refers to research papers. It adds to the
level of interest.

------
z3t4
If you want to build a "boat" you start with a canoe, you don't build a ferry.

------
ChemicalWarfare
googles and ubers of the world are not just their end-user facing apps + the
backend required to support them.

these are large businesses with end user-facing "IT" side of the house being
just the tip of the iceberg.

------
raverbashing
The question is why do someone needs the first 50Mi primes in a C++ file

~~~
ddorian43
See making crc 4x faster in redis by keeping precommputed values in a lookup
table: [https://matt.sh/redis-crcspeed](https://matt.sh/redis-crcspeed)

Maybe something like that ?

------
simonw
The code is the easy bit.

------
callesgg
Generally it is Marketing..

And also "Hofstadter's law"

------
clifanatic
> Businesses that actually care about turning a profit will spend a lot of
> time (hence, a lot of engineers) working on optimizing systems

Hm - every time I suggest optimization (of anything), I'm shot down with
"premature optimization is the root of all evil!"

------
mixedCase
I'm sorry for the OT, but I feel I should post this:
[http://bettermotherfuckingwebsite.com/](http://bettermotherfuckingwebsite.com/)

This is unreadable as is.

~~~
minitech
Resize your browser window?

~~~
Veen
Yeah, but asking readers to resize windows, resize the font, and tweak the CSS
to get decent line heights is a bit of cheek when it's trivially easy to
implement at least minimally pleasant-to-read typography and layout.

It's a bit shortsighted of a writer to completely ignore the realities of
readability. The current layout may conform to some vague idea of purity or
minimalism, but that puts form over function in a big way.

~~~
cpburns2009
Resizing a browser window is hardly as involved as tweaking CSS or browser
settings.

------
draw_down
Every variation of this comment/mindset drives me nuts. It's unbelievably
thoughtless.

