
Startup CTO – Premature Scaling - mkarliner
https://medium.com/@mike_93443/startup-cto-premature-scaling-cabf02c54060
======
grandalf
I've found that most startups fail to scale when it comes to making
fast/responsive product decisions.

This may not be obvious, since those startups are typically "agile" in the
buzzword sense.

The cost comes when the universe of possible change is constrained to the
point where the right questions aren't being asked because the answer would be
"no we can't do that". So the startup enters a dark age in which scientific
thought and rational problem solving isn't really relevant. Often at this
point the team starts using Scrum to force the business to plan better, when
the core problem is that the existing infrastructure is not able to support
the needed experimentation fast enough, making it useless.

This can happen at various stages of funding, since sometimes a series A or B
can occur without solid product market fit, or the company may be really good
at growth in spite of having a promising but mediocre product.

So I view the CTO's job as preserving the ability of the business to behave
scientifically and boldly until product market fit is found.

~~~
bwilliams18
I'd submit it's important to always be able to behave scientifically and
boldly. Product-Market fit isn't a one time solution. Markets move and change,
Facebook, Apple and Google have been successful because they've changed their
products as the markets have changed.

~~~
grandalf
Exactly. I totally agree, but typically with a startup achieving it at least
once (the first time) is of particular importance. After that the role is more
general as "CTO" than as "Startup CTO". But I'm just splitting hairs. You are
totally right.

------
exabrial
The problem is ego. "Scalability" is a sexy problem to have, when in fact,
"simplicity" and low-risk "trailing edge" technologies is the prescription for
success. Yep, i'm talking to you. You don't need an angular app backed by
MongoDB to display static content. Static HTML will work just fine.

~~~
korzun
Except that you don't need a CTO if you are building out static HTML pages.

Sometimes you need to put a twist on the problem to make it a bit more
interesting if you want to attract smart people.

~~~
nasalgoat
As a CTO I see it as part of my job to find the right tool for the right job.

Part of that was when I told the team I wanted our splash page and everything
on it to be just static HTML files that would be hand edited if and when it
was needed. There was no need for fancy CMS tools for a landing page that gets
updated maybe once a quarter.

So, that's part of why you need a CTO - to know when to use static HTML!

~~~
korzun
> As a CTO I see it as part of my job to find the right tool for the right
> job.

If that's a minuscule part of the job, then I agree with you here.

My point is, you are not a CTO if the technology behind the product is a
static web site. What do your VP and Directors do all day? What about the
teams under them?

There is a huge difference between a CTO and a front-end engineering.

------
bpicolo
One of the big ones: For a small startup, granular microservices are often
very premature, and will slow you down rather than speed you up.

~~~
jlebrech
write the MVP in Rails or Django and scale things that need scaling later.

you can scale the base app by giving it more hardware, but when you have to
batch process lots of stuff then think of services (most of the time you can
just split that to another server).

~~~
enraged_camel
>>write the MVP in Rails or Django and scale things that need scaling later.

There's a new alternative: write it in Elixir/Phoenix. You'll get the same
productivity benefits with none of the performance trade-offs. Elixir apps are
reliable and concurrent by default, and the Phoenix framework has none of the
"magic" that Rails has, making it very developer-friendly. :)

This isn't to say that Elixir/Phoenix are perfect. The ecosystem isn't as
mature as that of Ruby/Rails, so finding answers to questions can take a bit
longer. But that is changing quite quickly.

~~~
bpicolo
This is one where opinions are widespread and mostly the choice doesn't matter
so much, hah.

Fwiw, I've been pretty intrigued by the idea of dotnet on linux lately
(integrates with postgresql and everything). dotnet isn't exactly hip, but C#
is fast and it's mature (and has some pretty dope functionalish features. You
can run F# if you want to, too). Planning to try how well this works in
practice soon.

There are things I like a lot about Elixir, but also plenty that I'm not quite
a fan of.

------
1ba9115454
If I see premature scaling it's usually at the database side of things.
Inexperienced devs can be quick to dismiss a SQL database and go with
something based on key value pair caching.

So I agree with the article keep it simple and easy to implement, worry about
scaling later.

~~~
mkarliner
Yep. Not long ago I asked a dev why he had suddenly introduced Riak into the
system.

'Because it's cool'

Oy.

~~~
mgkimsal
Similar experiences. Worked on a project, and a 'lead' guy left with .. a few
weeks of a node/koa codebase in place that only he really knew. Between the
rest of us we had loads of C#, PHP and Java experience (multiples years each
over 3 people).

"Let's rebuild something based on what the bulk of us already know," I
suggested. (Keep in mind what we had was a few weeks of prototype code that
was not in production.)

"No, I really want to learn more node," was the only other input, and that
'won' the decision. 18 months later... "yeah, koa isn't really a goof fit for
our use cases".

~~~
softawre
This is where as a tech manager you step in and push people the right way. Fun
tech is fun, business is business.

C# is a great language, JVM is great w/ Kotlin/Clojure, both of those have
tons of web servers.

~~~
cutler
Whilst the JVM may be established, ie. boring technology Kotlin and Clojure
certainly aren't. It's now 8 years since Clojure 1.0 and there is a grand
total of 15 Clojure jobs listed on Indeed.co.uk, 12 of which are in London.
Kotlin boasts 2.

------
caseysoftware
Needing scalability means you _should_ have customers which means you have
revenue.

I tell every startup that I advise that "revenue solves many problems" because
if you have cash flow, everything else becomes easier.. you can buy spot
capacity in the short term. You can hire contractors to help refactor. You can
rent coworking space for an office.

There are many "good enough" solutions for now. Take advantage of them until
you _know_ what your next steps should be.. and then commit accordingly.

~~~
aries1980
This approach really lacks the product vision. Although I agree with it to
some extent, my experience is with semi-mature product, the code will stick
around for ~2 years on average. You need to be experienced to make
implementations that will be easy to replace when needed.

E.g. one of your assumptions there are contractors who can help you. That can
be true to a certain level, but even product vendors often lack the scale to
support their own thing above the basic levels (this is true for post-IPO
companies), now imagine this when you can't afford to incubate your product at
an expensive tech hub location.

Also, when you have cashflow, your business is under pressure to provide
better customer service and marketing, and this is where the things getting
tricky: you need to decide whether you want to grow your business or pay back
the technical debts. Because the investors priority is always business over
tech (rightly), you need to look 1-2 years ahead and make your key technical
decisions accordingly.

------
Ensorceled
The issue I keep encountering, again and again, is that you really have no
idea _what_ to scale until you need to.

For instance, I had a client where a very large online retailer wanted a
RESTful API into my client's data that they could call directly from their
customer's browser. Before the Black Friday Freeze. This was a completely
unforeseen requirement and the solution was NOT to scale the client's overall
website to handle the 5-10K requests per second but create a micro service on
a new stack that could.

Your startup will pivot, sign weird deals and change your target customer.
Optimizing for scale early doesn't help, getting to the point where you know
you need to pivot does.

------
bhauer
I am biased because in my work, I have indeed seen many projects that are
struggling or outright failing because of performance problems. The underlying
sentiment of this article may be (mis)interpreted by some as "don't worry
about performance." I feel that is not the right guidance, at least in many
cases.

That said, there is an important premise presented by the article. If I can
reframe it slightly: don't over-complicate your system architecture,
especially not needlessly. If, by selecting some fancy and trendy new
technology, you will complicate your architecture or development process for
unknown or modest performance gains, it's probably wise to postpone such
adoption until the need is more palpable.

My advise, again showing my bias, is quite simple: just select foundation
technologies that are mature and high-performance. Using high-performance
foundation software gives your application a high performance ceiling, which
in turn provides breathing room to build your application in a more "brute-
force" fashion. With this approach, you can defer performance tuning and
optimization for some time, and perhaps indefinitely.

I have seen plenty of small and small-medium businesses operate well with very
modest technology platforms. But importantly, they are modest in architecture
(few servers, few processes, low system complexity, simple deployments)
because they used high-performance platforms (e.g., JVM, Go, so on). On the
flip-side, I've seen many small businesses struggling and prematurely adopting
complex system architecture because their software platform (e.g., Ruby, PHP)
presents a bottleneck early in the business' life, which creates a challenging
scale event before the company has even found their market sweet spot.

When I've written about this in the past, I've pointed out that many people
call this situation a "good problem to have," which is of course valid in
important ways. But it's also crude and demeaning to your development team.
You're basically saying, "Fantastic, our business is scaling. Yes, our servers
are on fire and my developers are panicking, but that's great!"

It's better, in my opinion, to have used a high-performance foundation and
(potentially) avoid that unnecessary pain. All else being equal (e.g.,
importantly, your development team is comfortable with a JVM language, for
example), I recommend keeping _your application_ simple by using platforms
that give you a high ceiling to grow.

~~~
mkarliner
All fair points. I think the thrust of what I was saying is simply: resist the
pressures to provide huge scale before there is reasonable evidence that the
product has some traction. Some huge percentage of companies will have to do a
major pivot before they get that traction, so the balance should be heavily on
simplicity, speed and agility, and if that's at the expense of scale, so be
it. Obviously, performance _does_ matter, and so does good architecture, but
staying in business matter most :-)

~~~
bhauer
> _resist the pressures to provide huge scale_

You and I completely agree on this: _huge scale_ is a irrational target out of
the gate for most projects. The potential for huge scale preoccupies many an
architect whose aspirations exceed their budget, to say nothing of their
market reality. A premature preoccupation with huge scale tends to yield over-
engineered systems that are needlessly complex and work against all of the
goals you cite: simplicity and the ability to react quickly to changing
business needs.

It seems my addition here is reduced to this: All else being equal—if your
team is equally comfortable with two technology options—selecting the platform
with a higher performance ceiling will further defer the architectural
complexity typically associated with scale. Architectural complexity (e.g.,
multiple languages, lots of orchestrated or inter-dependent processes, many
abstraction layers, complex deployments) tends to be the friction force
against the simplicity, speed, and agility we both seek.

------
michaelfeathers
There's a myth around scaling [1]. Actually, it's almost like a cognitive
bias. We assume that we can usually use the same structure as we scale.
Because of this bias, we haven't thought enough about how to make bigger
structural transitions normal.

The GP points at the transition between the quick & dirty architecture and the
architecture that handles Google-scale, but those aren't the only good stable
points for a system.

[1] [https://michaelfeathers.silvrback.com/the-myth-of-
scaling](https://michaelfeathers.silvrback.com/the-myth-of-scaling)

------
gargarplex
Yes, but also understand explicitly what the scaling limits of your initial
solution are, communicate them to the CEO, and explain how long it will take
to build a scalable solution for the next level and how much lead time will be
necessary.

------
avenoir
While i absolutely agree with the author, how does this line of thinking
change when you start using AWS or something similar? I'm building a SAAS app
at the moment and a lot of these scaling decisions are incredibly easy to
implement without imposing any significant costs in terms of maintenance (and
money, a lot of times). This would've been a different story had i been using
my own infrastructure.

~~~
theptip
I think there's a key difference between building your architecture with
plenty of headroom to scale (leaving plenty of low-hanging fruit), and
actually implementing high-scale function before it's required.

In a cloud environment, it's quite easy to be comfortable that you _could_
scale out to 10-100x of your current capacity, if you have to. But it's
probably not worth proving that fact until you have some customers.

Or put another way, being in a very flexible environment makes it even more of
a no-brainer to delay building for scale, since you have so many free or
nearly-free options there.

Now of course, if you really do hit scale you'll have the fun problem of
needing to optimize your spend, but you can probably delay thinking about that
until well after you've proven product-market fit.

------
peterjlee
My answer to most scaling question is: by the time we have that problem, we'll
have a lot of customers and money so we'll hire smarter engineers with
experience in scaling to figure that out. I still make sure we don't do
anything stupid right now so the future engineers have more options on how to
scale. Also it helps that I'm not working on a zero-revenue consumer app.

------
HodGreeley
I think I can give one concrete example where a NoSQL choice (often) makes
sense. Common modern day scenario in mobile is to build an app and iterate.
Migration in RDBMS rapidly becomes a real pain. Worse, you have no good sync
solution in case you need it. Contrast with something like Couchbase Mobile.
NoSQL embedded db is much easier to get going with. (Just check the Android
docs on using SQLite and see how it compares to "here's my JSON, done".) Sync
doesn't get in the way, but if/when you need it, you're going to be really
happy it's there. (Disclosure: I work for Couchbase.)

------
chaostheory
Are people, who are just starting, still building complex chef and puppet
systems when Heroku and Elastic Beanstalk exist? .i.e scalability is easy now.
It's a click away or even automatic which these types of services. It's also
affordable.

~~~
korzun
> scalability is easy now

Dumb, often limited scalability is easy now. Elastic Beanstalk approach is
only as good as it's the weakest link, which you have no control over.

EC2 is having issues with recovering from standby? Good luck getting anywhere.
You can sit on the chat with the useless AWS 'technicians' that are paid to
waste your time until the issue is corrected.

~~~
chaostheory
I'm not sure if I'm making myself clear enough. I'm not arguing that this is
great for everything. In the context of the post, that's good enough for a lot
of apps just starting out WITHOUT the work of setting up something custom and
durable ala Netflix's infrastructure.

------
neeleshs
While it's true you don't need to design for scaling right off the bat, if you
do become successful and keep pushing out the scaling pieces , the cost of
scaling later increases nonlinearly. At some point it becomes so important
that you start losing customers and/or new business opportunities.

A CTO should be able to foresee this and act before scaling becomes a behemoth
project and no amount of money can buy time-to-market.

~~~
korzun
> the cost of scaling later increases nonlinearly At some point it becomes so
> important that you start losing customers and/or new business opportunities.

That simply is your opinion. You are more likely to lose customers for not
focusing on the core product first.

------
jlebrech
if you need to scale so much to be profitable you're doing something wrong.

except when you're selling an idea to a VC and scaling is all they think
about.

------
zelias
Wise words. In the earlier stages, you need to build a product that has room
to pivot rapidly for the next iteration of the business.

------
eddd
Just don't scale, until you are ready.

I highly recommend Eric's Schmidt take on this:
[https://youtu.be/hcRxFRgNpns?t=838](https://youtu.be/hcRxFRgNpns?t=838)

~~~
dboreham
Or "Only worry about solving problems you actually have".

Spending your own money provides good discipline in pragmatism, I've found.

------
halayli
> For the truth is that even the most modest technology stack will be more
> than enough to handle the traffic from the vast majority of startups.

It's not just the stack, the architecture matters more.

------
bedros
Simply, to counter the flood of new users, build a Dam called waiting list,
the slowly activate users as you build capacity

Put priority on paid users vs free, to cover server costs

------
romanovcode
In other words: Scalability is a good problem to have!

------
JMCQ87
Is it me, or is there not really much in this post which is not obvious, i.e.
common sense?

~~~
JonMR
Common sense is often worth repeating.

------
venture_lol
There is nothing wrong with planning for scale and for microservices or
whatever appropriate architecture from the start.

It does not take more time to design properly from the get go.

That, of course, assumes you have a CTO and a team capable of their job :)

~~~
dasil003
Yes, if and only if you know exactly what you need to build and how big the
scale will be from day 1.

What usually happens is the first thing you build is wrong, and you need to
iterate aggressively on not just the software but the entire business concept.
Once you have product-market fit, then you are in a position to design an
optimal architecture. If you go microservices before that, you are adding
unjustifiable overhead, which will inarguably slow you down compared to a
monolith when you are in the 1-3 engineer phase, and don't fool yourself about
that.

~~~
venture_lol
I frankly have no idea what you are talking about. You are making wild
assumptions that monolith are better just because...

It will all come down the the experience of the team, esp. the leadership -
the managers, the leads.

A strong, experienced, mature and effective leadeship will enable a team to
perform "magic" and set the right technology foundation that will prove its
worth as the business grows over time

~~~
taurath
The point he's making is the technology foundation can be built for the wrong
product or the wrong market. After 2 full product reconfigurations or pivots
requiring a TON of code being dumped you will wish you didn't have the
overhead of building infrastructure with the first product in mind. In the
end, the base reasoning for the foundations being built the way they didn't
hold up in production.

Building something quickly that can be broken up over time is the best way to
an MVP, and you can deal with scalability issues when you HAVE scalability
issues. I consider scalability to be a good problem to have, because by that
time you have actual traction.

If you are a bigco with a big marketing budget in a field in which you already
have experience and business then by all means scale to millions of users at
day one. Don't take on the cost of it in a startup, because the problem is
TRACTION not technology with regards to scaling.

~~~
venture_lol
You are making assumptions about both the premise and the conclusion :)

Designing and building for scalability, flexibility, availability, etc does
not mean going all out to boil the ocean and then find that you attempted to
boil the wrong ocean.

If that did happen, you got the wrong team :)

~~~
taurath
Sorry about the assumptions. It did happen, and it was the wrong team :)

------
partycoder
Just more advocates for hot potato engineering.

Build a snappy prototype that doesn't scale then sell it to some clueless guy.

