
Principles for Building New SaaS Products on AWS - jackgill
https://www.trek10.com/blog/guiding-priciples-for-building-saas-on-aws
======
ComodoHacker
This sounds like an excellent guide to a perfect vendor lock-in. It's just
missing the final principle:

#4 Operate as if you may be bought or cloned by Amazon at any time.

~~~
sl1ck731
Unless your entirely Kubernetes (and even then there are some sticky points),
using a cloud of any kind already has you "locked in" with some of the most
basic things like IAM. If you are going into a cloud you may as well reap the
benefits of the premium you're paying.

~~~
scarface74
I wonder has anyone who actually goes through all of the lengths trying to
“avoid vendor lock-in” actually sat down and done a realistic project plan of
how much it would cost as far as man hours and the amount of regression tests
that would be required.

Once you have any type of meaningful infrastructure you’re already not going
to migrate without major pain between data migrations, networking, security
and compliance audits, retraining, etc.

And if you are only using one of the major cloud providers as a glorified
colo, you have the worse of both worlds - you’re paying more for
infrastructure and your TCO is no lower because you’re still babysitting
infrastructure.

~~~
lostcolony
I want to know who has actually accepted a cloud vendor's lockin...and then
regretted it, had to change, and wished they'd done it up front. I have yet to
meet them.

~~~
streetcat1
Vendor lock-in shows up when you want to cut costs. For example, during a
recession.

It basically a fixed cost imposed by the cloud providers.

I.e. when the economy contract, and yet AWS/Azure profits are growing, this is
the pain of vendor locking.

~~~
scarface74
And employee’s salary is not a fixed cost? Or are you going to ask your
employees to do double duty because “we are family and we are in this
together”?

How many companies that were on such thin margins between surviving and going
out of business that cutting infrastructure spending by 10 or 20% was going to
make a difference in them being able to survive?

~~~
streetcat1
Yes, they are. But this is outside the point.

Which is, vendor locking did not show itself, because the economy was good.

~~~
scarface74
Companies have been “locked-in” to vendors since IBM in the 50s. There are
still plenty of successful companies that have been dependent on IBM for
decades and weathered plenty of recessions. That’s not to mention all of the
companies in the modern era that are completely “locked in” to the Microsoft
ecosystem and spend literally millions on Windows, Sql Server, Active
Directory, Office 365, etc. and have also weathered recessions.

Any large company depends on literally dozens of vendors that they have tied
their business process into where it would be a major pain to switch.

I know in the health care industry, a lot of health systems are so tightly
tied into their EMR/EHR, leaving a something like AWS would be a breeze.

~~~
paulryanrogers
Could it be large companies have war chests to draw on?

Smaller companies often run leaner. Except maybe for big airlines that spent
their surpluses on stock buybacks and executive pay increases.

------
taylorwc
I found myself nodding agreement until I reached the Cloud9 part... I think I
can count on one hand the number of professional devs I know that use Cloud9
as their primary editor/IDE. Curious if I'm in a bubble on that front?

~~~
shortj
Author here. HN in a lot of cases has readership that I’d put in the segment
of developers that should absolutely not be using Cloud9 daily.

I personally take using Cloud9 to the absolute
extreme([https://www.trek10.com/blog/i-buy-a-new-work-machine-
everyda...](https://www.trek10.com/blog/i-buy-a-new-work-machine-everyday)),
having my Cloud9 env setup scripted and creating a new one every day/project.
I don’t really recommend that approach for most folks. Anecdotally, it has
paid off well when I left a Mac on a train and I was able to walk in an apple
store grab a new one and lost minimal productivity for the day.

However, the flip side of all this is I regularly work with a lot of IT people
that have underpowered machines, flaky / poor internet, crazy restrictions on
their work machines that cause all sorts of problems with CLI / program
installation, etc. I’ve found Cloud9 to be super liberating for those folks
particularly with the parity of Cloud9 to AWS Lambda runtimes.

~~~
nicodjimenez
Thanks for the great content!

~~~
shortj
Appreciate you!

------
wiradikusuma
"I'd ask most developers to start their day in Cloud9" \-- hmm, no. Most
developers I know have powerful computers (many of them are gamers), so it's
wasteful not to maximize the ROI.

Also, not everyone has fast/reliable internet all the time :)

~~~
shortj
Author here. Similar reply to one I did below, but a significant amount of
developers I work with in enterprise or corporate contexts don’t have this
similar situation. Cloud9 can be fairly liberating for them short term,
especially while learning the ropes of AWS.

The majority of HN readership I’d encourage to continue using their own
tooling, you’ve got fast internet, unrestricted access and powerful equipment.

That all said, I default to Cloud9 these days just so I can bounce around
machines and have a consistent dev environment when I need it. A lot of my
daily job is meeting teams where they are and helping them be productive fast
as possible so I need to stay semi-fluent in most operating systems.

~~~
cj
Have you tried Linux Workspaces?

(Compared to Cloud9, I greatly prefer Workspaces, but still use Cloud9 on
occasion for a few niche use cases)

~~~
shortj
Yep! They work great in many situations. However, Cloud9 is quite a bit more
usable and stable on something like shaky/inconsistent airplane wifi. It’s
also way less friction to setup and tear down 3 or 4 Cloud9 instances in a day
compared to workspaces.

I treat Cloud9 like any other ephemeral editor process. Need a new editor
window? Cloud9 project. Done for the day? Commit everything I care about. Tear
it down.

That said, I frequently spin up Windows workspaces to test software or
workflows if I’m writing a guide or content.

------
jcims
These get a little boutique, but if you want to attract and close large
enterprise/regulated customers, I'd extend a few of yours:

\- Build your application to suit many customers or one. Large customers love
to have their instance run in a dedicated account.

\- Open source your operations. Large customers love to see logs/operational
activity from their environments (e.g. cc: cloudtrail/config/cloudwatch logs
to customer, this assumes dedicated account)

\- Open source your security. Be prepared to ship guardduty, config rules, etc
to your customer (this assumes dedicated account).

Also, since we're focusing on AWS here

\- Build your application to support hybrid cloud customers. Expose it through
private link, VPC connections, transit gw, firehose, api gateways, VPC
lambdas, whatever is appropriate for your architecture.

\- Leverage IAM as much as possible for authentication/authorization. Not AWS-
specific but implement SSO and assume customers will require numerous
instances of your service when developing your user account model.

\- Leverage KMS as much as possible to protect data at rest, and support
customer CMKs.

There are more but I'll stop here.

------
indymike
Some really good advice here. Especially the "build as if you are going to
sell at any moment" and "build as if you are going to open source at any
moment".

------
peterwwillis
Immutable Infrastructure is more important than Infrastructure as Code, fwiw.

The latter just means "it's in [version controlled] code". This has a variety
of use cases, and it might mean you end up in a quagmire of complexity. It's
become a cargo cult thing where I've seen people adopt horribly complex,
fragile solutions over practical ones "because IaC".

The former is a principle that basically has no downside, and only improves
operational integrity. Even if you're literally deploying everything by
clicking in the Console, it's still massively more reliable, repeatable, and
recoverable as immutable artifacts.

The next thing I'd recommend before investing heavily in IaC is auto-recovery.
The most obvious example is Autoscaling Groups, but any health check combined
with an automatic action such as restart or re-deploy can work. This works
best with Immutable Infrastructure, and typically does not require IaC.

------
jackgill
I'm interested in hearing opinions about this principle in the context of data
engineering:

> This also means leaning heavily into all the service offerings and
> orchestration tooling that is afforded to you by your platform.

I've built a data lake and several ETL pipelines using AWS native services
(Kinesis, Lambda, Athena). It works but it's a bit...fiddly. I spend a lot of
time configuring these services and handling various failure modes. I've been
wondering if I should be looking at third party vendors like Fivetran or
Matillion for ETL.

Does anyone who's worked with AWS data engineering services have thoughts on
the trade-off between AWS native services and third party vendors in this
area?

~~~
ramraj07
I can strongly attest to snowflake. I regret AWS doesn't offer the same
features without making us jump through the maze of services with which we can
emulate the same concept.

~~~
jackgill
Thanks for sharing, I've heard many good things about Snowflake. In the past
I've seen them as more of a Redshift competitor (data warehouse, as opposed to
a data lake) but if they can simplify data ingest then I am definitely
interested.

~~~
ramraj07
They're fundamentally different only in the model of decoupling storage and
compute completely, but in a far more simpler way than redshift spectrum I
feel like. Some of their features like zero-copy-clone are just not possible
in AWS and make it extremely simple to do pipeline management in way that (at
least to me) makes the most sense.

It's also the most democratizable model I have seen - anyone who knows the
slightest amount of SQL can be set up to explore the data in minutes.

The elephant in the room is that you need to use SQL. Their spark connecters
are as of now useless, so you either have to go with DBT, some homebrew SQL
stringing mess or something like sqlalchemy. We're currently developing some
wrappers around sqlalchemy to make this a bit less painful, but it's still so
worth it.

------
the_resistence
Thanks for this perspective. It really helps the noobs trying to build more
than single user stuff.

------
gfodor
What’s the right granularity to shard aws accounts? I haven’t gone down this
road. Is it madness to consider this as a multi tenant mechanism vs the usual
foreign keys in the database approach? Is applying cloudformation across
thousands of accounts feasible?

~~~
shortj
Probably the most useful mechanism I have for determining this is “if this AWS
account disappears, how screwed am I / can I recover.”

I tend to separate all of my projects/services, and each of those to
environments.

A cold storage AWS account, audit and security (ship logs, config changes,
etc), shared services to another account.

If dev account gets hacked, that sucks, but we can clear it out.

It prod gets hacked (and deleted!) that super sucks. But hopefully cold
storage and audit accounts can help us out.

If some other services/projects account gets hacked, I don’t want to be
worried about impact to unrelated projects.

~~~
gfodor
Nice approach - for cold storage what do you mean exactly? Manually rsynced
backups or something? Most aws services I’ve used that have backups built in I
don’t recall having cross account writability.

~~~
shortj
RDS snapshot copying, EBS snapshot copies, S3 cross account bucket
replication, etc. Write only with no entry points into that account from your
other accounts. (Preferably its own locked down IAM role with MFA required)

~~~
gfodor
Cool- stealing this, thanks! Do you do the backups w a scheduled lambda?

------
ignoramous
> _For instance, AWS doesn 't have anything quite as tuned to fast frontend
> search experiences like Algolia._

[https://aws.amazon.com/kendra/](https://aws.amazon.com/kendra/) ?

~~~
shortj
From my understanding when I first read through Kendra info (albeit that was
launch day) it is more enterprise knowledge-base search and not quite the same
use case as Algolia.

~~~
1cvmask
I understood the same.

------
eugenekolo
Not sure how much I can trust "AWS Gurus" about actually building SaaS
products in a real world at a real company.

It's all good advice, but please provide me with the money, and resources
necessary to do it all.

------
simonebrunozzi
What a lame article. I don't find any utility in any of the principles
expressed here. Also, in my experience (at AWS 2008-2014, before/after in the
same industry), I can't recall instances of companies that either mentioned,
or followed, these principles.

Specific critiques:

> #1 Build as if you may sell at any time

> ... it forces you to build with best practices and isolation.

The opposite. It gives you an incentive to postpone technical debt, you want
to grow and be acquired at the expense of whoever is going to integrate your
startup into $bigco later.

Side note: "AWS Organizations" to me is simply a way for AWS to try to cover
for the poor design choices of the organizational structure of an AWS account,
and the unnecessary complications related to billing and metering.

Try to understand the AWS bill of a sufficiently large company - you won't.
The AWS rep won't. The AWS Solutions Architect won't.

Also, never heard of a company being acquired at a higher price because they
had a "proper" AWS setup.

Ah, forgot this: if your acquirer is using MS Azure, good luck telling them
that you are using Cloud9 or other AWS-specific stuff.

Let's continue...

> #2 Build as if you may open-source at any time

Yes. In an ideal world. In practice, almost nobody follows best practices.
Because there's always some urgency that takes precedence. This is why
companies like Accenture, PwC, Deloitte, etc, keep billing monstrous amount of
money to help large companies "migrate" or "evolve" or "adapt" or whatever
buzzword they use.

> #3 Build with a cloud-native mindset

> ... going outside of the platform should be an exception and something you
> do only when truly needed

> My thinking on serverless these days in order of consideration. > \- If the
> platform has it, use it > \- If the market has it, buy it

Serverless, really? A promising, cutting-edge technology, sure. You want to
bet your startup on Lambda? Go ahead. Lambda has been around for ~6 years now
(I even tried a super early version internally before it was released), and I
still haven't seen a large company doing A LOT of development on Lambda. Most
project using Lambda are small, confined, isolated projects and/or teams.

Cloud native is great, WHEN it makes sense. I don't like religion too much,
and I don't like religious people either; the author seem to have taken his
faith in AWS too seriously.

------
Mizza
This is blogspam.

~~~
unethical_ban
It isn't blogspam just because it doesn't interest you.

~~~
Mizza
It's a "3 tips" promotional listicle article on a company's website. It's the
very definition of blogspam. I am interested in the top which is why I
clicked. It's blogspam.

