
You are not Google. You are not Netflix - dimm
https://programmerfriend.com/not-google/
======
alfiedotwtf
I worked on the floor with two teams. One (ours) built our apps and
infrastructure the correct way. JIT. Get it working, get it working correct,
get it working fast. And given the scale, it did wonders.

The other team talked big and gave demos to higher ups and their higher ups
and their higher ups. Web scale baby. And with that came the galactic
infrastructure and all the buzzword needed in order to run it.

About 6 months later, they quit. They had spent their whole budget on
astronaut architecture, that the didn't have any money or time left to build
the apps that were supposed to run on said infrastructure.

"You are not FANG". But even if you are, JIT development.

~~~
islanderfun
I don't believe I'm familiar with JIT in this context? Just in time
development?

~~~
OJFord
I haven't heard it either, but apparently it's another name for the Toyota
system/'lean manufacturing': [https://en.m.wikipedia.org/wiki/Just-in-
time_manufacturing](https://en.m.wikipedia.org/wiki/Just-in-
time_manufacturing)

~~~
alfiedotwtf
Sorry. No, I wasn't referring to JIT Manufacturing...

It was merely a tongue-in-cheek to mean just build what's needed right now in
front of you, then get it working at scale once there's an actual needs to
scale. This is in contrast to starting from day one spending most of your time
thinking of scale, then building everything so that it can infinitely scale
(along with the complexities that come with it), only to find out that you
have 8 users and your data can fit entirely in memory.

~~~
mgkimsal
there's a vast middle ground between 'astronaut architecture' and 'common
sense necessities', and knowing where a good cutoff point is is hard without
experience.

I've met more than a few folks who consider web templating systems to be
'overkill', and have 0 understanding of the risk of xss. Same with sql/db
escaping - "I just write the SQL and run it, using all those libraries is just
a waste of time", etc.

Many of the projects I've come in to over the years were doing with a "JGID"
mentality. And they did "get it done", "it" was just a steaming pile of crap
when it was done. "Why is this taking weeks to do - the previous guy was so
much faster?"

Had this one last year:

"This was never slow before when X worked on it, you've made changes what have
you done? We need to call X back in to the project".

X just "got it done". And X was a db admin who was writing code. And X decided
it would be good to have views join against other views which joined against
other views, and have some queries which used those views run in triggers.

When X was on the project, and there were 40 users, it was fine. They hit 1000
users, and things were 'slow', so they upgraded to a larger EC2 instance. X
left, and I came in, and several months later they hit ~30000 users (not
active, just ... user account records). The system was dying with more than 5
active users, because of all the views joining other views on 30k+ records.

Unravelling that meant deciphering all the views, all the queries, all the
code that touched all of it, and rebuilding a moderate portion, without tests,
known 'good' data, or anyone on the project knowing what 'right' was - they
just knew when things looked 'wrong' (or slow).

BUT... 18 months earlier, it was "JGID", and it got done. I'm still a bit
perplexed why a professional DB admin thought views joining against multiple
other views was a good approach.

We've all got horror stories, I'm sure, it's just that the "JGID" mentality is
often preached by competent experts as a good approach, but picked up by
beginners as the approach used by experts, and has bad consequences.

~~~
dhuramas
Out of curiosity - Which database is this? and is it really taboo to have
views joining against multiple other views?

If the underlying views were performant- I'd assume the query optimizer would
do the right thing(at least 90% of the time).

EDIT: I guess it depends - Just did more research and found this [1]. As long
as the views don't do unnecessary heavy lifting or joining unnecessary tables,
it should be fine.

[1] [https://dba.stackexchange.com/questions/151169/are-views-
har...](https://dba.stackexchange.com/questions/151169/are-views-harmful-for-
performance-in-postgresql)

~~~
mgkimsal
postgres. I realize this is heresy to badmouth Postgres here, but it was
Postgres (9.0 or 9.1 IIRC).

Had a query that selected from table X joining against view X which selected
from view A, and view A joins against tables A,B and C, and table B also joins
against a view which uses table A and C.

This was just bad. But trying to explain to non-tech people how bad it is,
when "it used to work", is difficult. It used to work when you had 50-100
records. No one ever tested was this would be like with 30k records and 50
concurrent users all executing the same nested/circular view mess
simultaneously.

But the fact that it was postgres is kind of beside the point. I don't know of
any mainstream DB that would handle this well.

The short term fix was to do this large query once at the end of a process and
cache the results; the set of queries in question were happening on a
'dashboard' view which everyone hit all the time. It would still cause
problems with concurrency, because when 80 people would go through a process
and get 'done' (think timed training exercises), the queries would still all
be running more or less concurrently, and still cause timeouts, but it wasn't
_as_ frequent, because people tended to be staggered a bit more as they
finished.

~~~
jacques_chester
PostgreSQL has progressively reduced the number of optimisation boundaries
that are encountered with views-on-views situations, but you can still wind up
telling the database to churn and slosh a whole bunch of data from which you
only cherry-pick a tiny portion.

I have definitely been guilty of failing to test how my schemata behave with
large data sets.

Oh then there's ORMs. I've seen ActiveRecord spit out some frankly batshit
insane queries that would stump a room of Einsteins. But somehow PostgreSQL
picked it up, chopped it into a plan and got to work plowing through an
incredibly wasteful and repetitious query.

~~~
mgkimsal
> I have definitely been guilty of failing to test how my schemata behave with
> large data sets.

As have I. It's the dangerous part of the "get it done" approach. And there's
no perfect approach - everything is a tradeoff. How much time do you spend
dealing with situations that might never happen?

Experience does give you some grounding when making those tradeoff decisions.
No, we don't need the architect the application to scale up/down to handle 25k
concurrent users in 5 minutes; that's unlikely going to happen. Yes, we do
need to spend the extra 2 hours installing and learning a templating system to
avoid common XSS pitfalls.

------
empath75
I don’t think one needs to plan on being Facebook scale from the beginning,
but I do think there is some consideration of scalability that needs to be
considered from the start. Maybe you don’t need to think 10 steps ahead but
you should at least have plans for 3 or 4 steps ahead of you don’t want to
have to rearchitect everything from scratch in a year.

It might not be strictly necessary to containerize your app, for example, if
you’re just running it on a single ec2 instance behind a load balancer, but
it’s not a huge amount of work up front and there are some benefits and it
means it’ll make it easier to move to ecs or k8s later if you need to.

~~~
geggam
Thing is if you want to scale to FB size you _have_ to re architecture /
redesign.

No one starts an application with the farm model and multiple colos.

~~~
toast0
Starting with two colos might be a good idea -- at least at the point where
you start thinking of adding a second machine for data availability, it may as
well be in another colo so you have a disaster recovery option.

Starting with partitioned data is a lot bigger jump. At most, I would say
avoid building features that would require heroic efforts to implement in a
partitioned world, but a single database can get huge these days, and maybe
even huger by the time you need to scale. Chances are, 2TB of ram and 64 cores
will fit your live DB needs; and if not, you'll know with time to make
something else work

~~~
geggam
Start with a hosted DB service in the cloud. When you outgrow that then start
thinking how to scale. 1st start sharding the data, then think about splitting
a few services out if they are changing faster.

This concept we need to go full microservices multi region right from the get
go is silly and wasted time / effort.

Until your idea sells and your market grows there is no reason to concern
yourself with the logistics of scale.

------
CM30
A related issue is also focusing so much on the tech (and often how well the
product/service can scale) that you forget to do much marketing and end up
delaying the product/service for months as a result. Or worse still, never
actually finish building it at all.

Definitely follow the old get it working, then correct, then fast methodology,
at least for projects without many billions of dollars in resources.

~~~
zzzcpan
> follow the old get it working, then correct, then fast methodology

And eventually realize there is no correct then fast after get it working,
only more iterations forever.

------
sanderjd
I think this article is basically right, but I want to provide a personal
perspective on this:

This is why I have found that I prefer working on a widely used product at one
of these big companies. I find the most enjoyment in thinking through how to
make products work acceptably for huge numbers of users, or how to pick the
needles I need out of enormous data sets, or how to structure a codebase such
that hundreds of people can contribute to it every day. I have found it very
demoralizing in my career when I've been told not to think of user or data or
codebase scaling considerations because YAGNI. I've lost a number of debates
against just-get-it-done ethos, and mostly I think I was right to lose; I was
wrong, YAGNI was right. But the rightness of YAGNI for those products didn't
make me enjoy the work any more.

So I think the article is right that most companies don't have these scaling
problems, but for me, I want to work for the ones that do. I think this is
probably a common sentiment.

~~~
vinay_ys
I feel the same way. I have always sought to work for such start-ups that are
about to take off and do high-scale; where leverage is important. Leverage
provided by code-bases where certain changes to the code makes others working
on the same code base 10X more productive; leverage provided by org designs
where changes to the org-design can make all the roles 10X more productive.
Looking for these leverage enhancing opportunities and making it happen is a
very satisfying and unique experience.

But there's always YAGNI of some kind in any situation and I'm good with the
kind of YAGNI that these scaling situations have.

------
nemild
Two articles on this topic:

You are Not Google (my favorite): [https://blog.bradfieldcs.com/you-are-not-
google-84912cf44afb](https://blog.bradfieldcs.com/you-are-not-
google-84912cf44afb)

A piece from my Notes to a Young Software Engineer that likens engineering
media to a bazaar, not a mirror of the industry:

Beware Engineering Media:

[https://www.nemil.com/on-software-engineering/beware-
enginee...](https://www.nemil.com/on-software-engineering/beware-engineering-
media.html)

(Martin Fowler's YAGNI article is also great)

~~~
tim333
The first one got 572 comments recently
[https://news.ycombinator.com/item?id=19576092](https://news.ycombinator.com/item?id=19576092)

------
mathgeek
> A developer’s skill is measured in one parameter: how well he writes code.

While this could be a valid viewpoint, one thing I’d caution against is that
this type of measurement inevitably leads to the commoditization of
developers. Rating skill on a single point will lead to “you’re only worth
what this metric says you are worth” and eventually to constant competition
between developers instead of cooperation. At least that’s the experience I
had in such positions during my younger years.

~~~
booleandilemma
Actually I’ve found that a lot of development is actually about image
projection, or how good you make other people think you are. This is
especially true with non-technical folks, who don’t know 1/2 of what
developers do anyway.

Our office’s “rockstar” dev has his 3 widescreen monitors tilted vertically
and he uses a split keyboard. He’s so quirky and leet.

------
vinay_ys
There's a lot to be said about org design. Org design has to be right for the
size of the company and the velocity you need to succeed. For example, when a
start-up that is about to take off like a rocket ship, it needs to make a
conscious change in org design to facilitate both strong foundation building
as well as fast feature building with a layer of domain-specific
platform/frameworks being curated to keep the leverage high and hence velocity
high without blowing the headcount budget.

Ideally orgs should sow the seeds of this 3-layer org (and corresponding tech
architecture) design from the beginning and consciously choose to scale the
investments in those layers as needed in each growth spurt.

Usually people tend to shortchange this advice by thinking they can make the
same 10 people do the work in all the layers. And that may be ok when you are
small. But specialization in systems vs app-frameworks vs app feature
development is highly important for velocity. Also your hiring loops and the
"hiring bar" has to be different for these roles. And usually the hiring
pipeline depth and churn rate is different for these roles. So, creating a
structure where impedances are matched correctly between input and output, so
to speak, is immensely helpful for achieving high productivity and cost
efficiency.

------
altitudinous
Its all about execution and delivery. I'm pretty happy to have left the old
days of corporate committee projects behind to do my own stuff. I'm also
financially better off.

------
DyslexicAtheist
Kubernetes, Machine Learning, Microservices, Service-Mesh, are the things that
immediately come to (my) mind from such discussions with start-up dev-teams.

But somehow all of these are at least a few years old meanwhile. Could it be
that the buzzword bingo we experienced in the last years has already peaked or
why haven't we seen any new nonsense[1] (at least nothing that I'm aware that
is not at least 24 months old)?

[1] it doesn't have to be nonsense but as the article suggests, not everyone
is FAANG.

~~~
geezerjay
Machine learning may have been used and abused as a meaningless buzzword but
kubernetes in particular and container orchestration in general is a well
established and proven infrastructure technology. If you are not operating a
monolith and you need to automatically deploy and scale your service then you
have no technically-sound argument to not use this sort of infrastructure.

~~~
DyslexicAtheist
the question is if an early stage start-up with 4-5 devs where they typically
already struggle to get basic CI in place ... and where 1 of the devs loves K8
and the others don't really know it (but don't object because it's nice to
have it on the CV), whether they should really already focus on orchestration
and scaling before they even have a MVP.

I'm not against k8 per se, but its complexity isn't something to ignore. It's
all good if you have the people to run this but most places I know don't.

I think we have reached peak k8-hype last year when F. Campoy at GOTO
suggested to wrap K8 around GO to make Go as resilient like Erlang/BEAM (see
video below ...)
[https://www.youtube.com/watch?v=ScE9TnoWltA&t=1437s](https://www.youtube.com/watch?v=ScE9TnoWltA&t=1437s)

as for ML I really think there are cool use-cases, but why would a pre-MVP
start-up already be talking about hiring data scientists, ... when they don't
even have any data (exception being they are an analytics start-up obviously).

this is how you get ants.

~~~
geezerjay
> the question is if an early stage start-up with 4-5 devs where they
> typically already struggle to get basic CI in place ...

Nowadays it's super easy to setup a CICD pipeline. The basic settings and
intro examples alone are enough to build and publish container images to a
container registry.

> and where 1 of the devs loves K8 and the others don't really know it (but
> don't object because it's nice to have it on the CV), whether they should
> really already focus on orchestration and scaling before they even have a
> MVP.

That doesn't make any sense. Developing services to run on kubernetes is a
software architecture problem, one which only affects some minot dev level
details such as supporting env variable settings. Devs can be entirely
oblivious to kubernetes and still be effective.

> I think we have reached peak k8-hype last year when F. Campoy

That's largely irrelevant and misses the whole point. Just because some people
present some gimmicky take on a technology it doesn't mean the fundamentals
are not good. Kubernetes allow distributed services to be deployed
transparently across multiple providers (thus you are not held hostage by a
seller), and allow services to automatically scale horizontally with demand
and within resource limits gracefully degradate after that point. That doesn't
go away if someone decides to reenact Inception with buzzworthy technologies.

------
wayoutthere
Having seen the inside of both good and bad product organizations, culture
beats process any day of the week. Get the core values right, hire smart
people and everything will fall into place.

A corollary is that cargo-culting well-developed processes generally means
you’re copying from a company with a culture that couldn’t address those
functions organically. So you may actually just be lifting an over-engineered
process designed to address a cultural weakness at the original company.

------
spyspy
Because we’ve railed on “programmers” for so long everyone wants to be a
“software engineer” which more often than not leads to equally insane
architectures. No you don’t need multi-region availability. No you don’t need
to rewrite this in rust. No you don’t need a multi-master db setup. No you
don’t need graphql.

~~~
xenospn
These weren't even "things" 5-7 years ago. When did it all change?

------
odomojuli
One impression that occurred to me from reading this article, is this excerpt:

"Why try to be like others if you are condemned to being yourself" Link:
[https://mutterichbindumm.tumblr.com/post/92142897112/funeral...](https://mutterichbindumm.tumblr.com/post/92142897112/funeral-
march-for-ludwig-ii-king-of-bavaria)

------
im3w1l
What's up with the title?

~~~
stevenjohns
Looks like it references the wrong article? This[0] one has the same name as
op's

[0] [https://habr.com/en/post/456568/](https://habr.com/en/post/456568/)

~~~
balfirevic
That was much more interesting article than the submitted one.

~~~
missblit
As an old(?) person who does everything by the book in C++, I couldn't really
relate to it.

At least at my company it's hard to get hired to a programming position on
soft skills alone. I've interviewed some very nice people who ended up not
getting hired due to trouble whiteboarding.

As far as I can tell no one I work with is a an influencer on insta-twit. Some
coworkers are better than others, but on the whole they all know the front end
of a for loop from the back and are willing to learn.

And I end up spending way more time not writing code than writing it.

