
AWS services to avoid - kiyanwang
https://medium.com/teamzerolabs/5-aws-services-you-should-avoid-f45111cc10cd
======
antirez
"Remember redis is meant to hold ephemeral data in memory, so please don’t
treat it as a DB. It’s best to design your system assuming that redis may or
may not lose whatever is inside"

Of course I disagree with that statement. Many serious Redis use cases at big
companies assume Redis is there and will hold your data. What they usually
assume is that under certain cases during failovers or other serious failures
you many lose some acknowledged writes that were sent immediately before. And
this most of the times does not happen too. Note that many high performance
applications using SQL DBs with a relaxed fsync configuration will have the
same assumption. The suggestion should be more: if Redis for you is just a
volatile aid, use a given setup, if it's were you store data, use another
setup.

Anyway if you want to see a version of Redis where you can store also bank
accounts, transactions or any other of the most critical stuff you can store
in a DB make sure to check the news at Redis Conf 2020 (the conf is in
streaming and free).

EDIT: But even after that announcement the biggest value for Redis is that it
can be fast and provide a "best effort" consistency level that is adequate for
a lot of use cases in practice. Many systems were sacrified in the temple of
wanting to provide linearizability for all the use cases.

~~~
ComputerGuru
I was agreeing with you until

> And this most of the times does not happen too.

This rubs me very very wrong. There is no maybe in data integrity. Redis
wasn’t designed to be ACID and shouldn’t be treated as such just because
“usually” it doesn’t lose data “under the right circumstances.”

If a sysadmin uses an RDBMS with relaxed fsync, they know what they are
getting (and durable writes is not one of them, no usually about it). Same for
Redis.

~~~
masnao
> sysadmin

we live in the age of devOps. there's no such thing as sysadmin, or god
forbid, DBAs.

/sarcasm

~~~
ramraj07
For real! Where did all those DBAs go??

~~~
lykr0n
They're charging $400 an hour to come in and unfuck years of devs thinking
they can design databases because they read a MySQL How-to Article

------
languagehacker
This is definitely a post based in experience, but maybe not based on broad
expertise in using these tools.

I don't want to pick apart the entire post, but I will say that the Lambda +
API Gateway example is maybe the best example of jumping into the cloud-native
world with blinders on. Just about every modern programming language has a
toolset right now that will do the work of generating a CF template for you
that creates an API gateway that forwards all HTTP routes to a single Lambda,
and then that Lambda is responsible for handling the actual routing of that
request. Examples include Zappa, ClaudiaJS, and Ruby on Jets just to name a
few. I can't imagine providing a feature-rich web application in Lambda
without this kind of abstraction.

Not knowing that such tooling exists, or explicitly choosing not to use such
tooling, and experiencing pain as a result seems to be common theme in this
article. If you dive into architecting a system using AWS's product offerings
without understanding the tradeoffs you're making, you will experience greater
cost and greater friction -- hands down.

~~~
dayjah
Yeah, missing the proxy pass option will definitely burn a prospective Lambda
user.

I also felt a similar tinge when the post talks about KCL. Yes, you use it,
and yes there are rules about output streams - but they all make sense when
deployed in fairly complex environments. I can’t remember when I last used
stdout for logging outside of greenfield development. Once it’s on prod
everything goes to stderr and is highly structured.

Similar sentiments about Cognito. At some point of complexity you must have a
server in between using the Admin* range of commands. If you want their “get
going quickly” then yes, there are trade offs like WebViews. That said; I’ve
never used Auth0 - so insert a Luddite warning here.

Finally, one hundred percent agree on CF.

~~~
redisman
Another pretty basic Lambda thing the author didn’t mention is the alias
feature. So rather than having users-api-dev, users-api-qa - we just have
users-api and it has a alias for each environment. The environment variable
management is not amazing but just using a key-value JSON {dev: {a: “bc”}}
works fine.

There’s also truly no reason one lambda function can’t have a collection of
APIs (using the proxy method from API gateway). It’s literally no different
from any other micro service setup. You can pick any abstraction for a
function to cover.

~~~
Aeolun
If you start trying to work around the limits of your abstraction, maybe you
just chose the wrong one in the first place.

This is what seems to happen when working with Lambda for me.

------
ufmace
I have experience with a few of these. For those, the comments seem kind of
"duh" quality.

So you say Kinesis is bad for message streams where each message should be
handled by only one machine? Well duh, that's how it works. Kinesis is meant
for every listener to be assured to get the entire stream. It was never
designed for that, and I expect you'll have a bad time if you try to use it
that way. SQS is obviously what you want if you're interested in guarantees
that each message is handled once and only once.

One Lambda per route in a decently-sized restful webapp is ugly and
unmanageable? Well duh. Don't do that. Do pretty much anything else instead.
Seriously, anything else at all.

This sounds more like, if you want to deploy services on AWS, do either pay
somebody who knows what they're doing, or spend some time checking out the
services to make sure they are designed for what you want. Don't just pick a
random AWS service and usage pattern from a Google search and start building
around it without ever checking if it matches what you're trying to do.

~~~
ignoramous
> _SQS is obviously what you want if you 're interested in guarantees that
> each message is handled once and only once._

You mean SQS FIFO? SQS classic cannot guarantee exactly-once delivery or
ingestion, either due to failure at the client or server. SQS FIFO, however;
can but requires both the client and the server to work in-tandem to ensure
exactly-once processing.

~~~
awinder
And your queue throughput also gets destroyed by flipping that FIFO option,
it’s a mess.

How’s amazon’a kafka offering going?

~~~
isugimpy
I'm not a fan of it, personally. The interface is lacking, configurability
after launch is minimal at best, and the observability is EXTREMELY poor
because you can't get per-topic metrics without paying extra.

------
watermelon0
> With lack of drift detection comes great uncertainty.

But CloudFormation does have drift detection:
[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/detect-
drift-stack.html)

I'd pick Terraform before CloudFormation, but there are things that are better
with CF, such as rollbacks (they really do work as expected in almost all
cases), and AutoScalingGroup's UpdatePolicy (rolling/replacing deploy of new
launch configuration with healthchecks), which is not available outside of CF.

~~~
dastx
There are a lot of issues with both cfn and tf. As always,it comes down use-
case.

In my opinion, tf is terrible choice unless you're going to be using it for
more than AWS

~~~
acdha
Or if you want to use AWS features on a deterministic timescale:
CloudFormation frequently lags on support for new features - I’ve worked on
multiple projects which used Terraform for months before the equivalent
feature could be used in CloudFormation. For example, when ECS launched
secrets support it took most of a year before CF was updated.

This is a big deal with classic CloudFormation because there was no escape
hatch. Terraform can run arbitrary code if needed but you had to split CF
stacks apart so you could have something else run in between. That’s probably
better with CDK, of course.

~~~
synthc
CloudFormation has an escape hatch via custom resources, which can call Lambda
functions do create/update/destroy resources. It's ugly, but it works.

~~~
dastx
At my old place we had some automation around this. Essentially our custom
resources was mostly just the python scripts and then any deviations from the
basic custom resource definition. Made life a lot easier.

------
KaiserPro
Cognito:

yes, its a complex poorly documented pile of shite. BUT. It does work as a
reasonably secure OAUTH2 thingamebob. However I was told by my AWS account
manager that auth0 was the way forward, and I agree.

Cloudformation:

Meh, I have about 35k lines of active CF at the moment. Its much of a
muchness. Unless you are using parameters with selectors, you are going to
have a bad time. Hard linking templates together (I assume thats what nested
stacks are) is terrible. I've only briefly used terraform, so I have no idea
if its much better.

CF _could be_ a lot better. Like compile time validation, not just in time.
that would stop a lot of anger when you realise you've spelt a CF parameter
wrong(or the value fails validation) but only after you've spent ten minutes
for it to spin up. Thats frankly unforgivable.

Elasticache:

Yes. Its expensive.

KINESIS:

What a disappointment. Stupid naming conventions, Terrible throttling and
throughput. Its just horrific. Whats worse is that they looked at SQS and
thought: "this compares favouribly" NATS.io is a great fit for certain
usecases (no, kafka is never the answer)

LAmbda:

I don't actually get this myself. I made a REST api exclusively in lambda. It
meant that I could build a working prototype really quickly. Once proven we
ported it to fastapi in an autoscale group.

The API gateway was heavily integrated into the lambda spinup (controlled in
CF) so I really don't see what the issue it. Also it understands swagger, so I
struggle to understand the criticism

~~~
swyx
seriously, an AWS account manager told you to use a competitor? is this
common? in a normal company salespeople might lose their jobs for saying that.

~~~
mcherm
> in a normal company salespeople might lose their jobs for saying that.

Quite a few times over my career I have had a salesperson (not from Amazon)
recommend a competitor over their own product. In every single case my respect
for the salesperson shot way up. In at least two cases I can recall this
helped them close a sale.

A smart salesperson does not do everything possible to push their company's
products... a smart salesperson solves their customers' problems.

~~~
zdragnar
> A smart salesperson does not do everything possible to push their company's
> products

Bingo. A bad product fit means a bad customer experience, which means a bad
review or reputation.

The smaller the company, the more important referrals are from your customers.
Sending a potential customer to a competitor will (potentially) earn goodwill
and future referrals. At worst, they might not refer anyone your way, but at
least they won't be badmouthing you either.

Unfortunately, large companies typically mean large customers, and the people
with the buying power aren't the people who will be using the product... so
neither party really cares all that much about how well the product fits. This
is the old "nobody gets fired for choosing IBM" mentality.

The worst is when medium companies think they are big companies, and try to do
that to small customers. I once saw a salesperson push hard for something that
was very obviously too small to be worth our time, and the project management
overhead would have lead to blowing our potential customer's budget out of the
water. In the end, they walked away without working with us, and a pretty sour
taste in their mouths from the pushiness of the sales guy.

------
ashtonkem
The section on Lamba missed the biggest problem: latency and cost.

For low regularity calls, Lambda suffers badly from the cold start problem.
The first call to Lambda must actually create the lambda which can take
hundreds of milliseconds. This problem also shows up when the level of
requests exceeds the number of lambdas currently active, causing a cold start
as the new lambda instance is created. It’s not uncommon to see some services
inject fake requests with some regularity to insure that there are enough warm
lambda instances to avoid the cold start problem, which is silly.

Secondly, all lambda calls are charged by the millisecond, but the all calls
are rounded up to 100ms. So if your typical lambda call is 5ms, you might be
paying for 20x more time than you’re actually using.

Both of these issues led my former team to use a regular ECS app rather than
lambdas.

~~~
bastawhiz
Your mileage may vary. I process tens of millions (if not hundreds of
millions) of requests on Lambda each month. If you have meaningful volume,
cold starts aren't a problem. And when with my volume of traffic, lambda use
is a tiny fraction of my AWS bill ($30 max?). I'd be interested to know who is
dissatisfied with the cost and paying more than $100/mo and what you're
running (and how that would be expensive for you, considering the kind of
operation you must be running).

But also, if you don't have highly variable traffic, why would you use Lambda
in the first place? If you have negligible traffic (enough to sit on the free
tier), why not just use a single cheap EC2 instance? Lambda trades start time
for lack of a server—it's shared resource utilization taken to the extreme.
You're lowering your cost by letting AWS use the bare minimum to keep your
service available, and that means turning your code off when it's not running.
If you want to keep your code available at a hundred millisecond's notice,
just have a server running.

I assume most of the folks running into this just couldn't be bothered to pay
the $7/mo for a hobby Heroku dyno or run a dirt cheap EC2 instance. Really
interested to hear from folks that find Lambda impractical for serious use
cases.

~~~
jeremyjh
That doesn’t sound right. A hundred million requests would cost $350 a month
just in API Gateway charges.

~~~
empthought
That's REST requests... HTTP requests are $1/million.

~~~
ashtonkem
That’s still $100, which is much more than the $30 mentioned.

------
katzgrau
I thought for sure I was going to find Elastic Beanstalk. It's great when it
works, but when it doesn't, the list of ways it makes life hell is a long one.

Elasticache though? Pricey, but I recently got an alert at 1am that one of my
core services was down and it was because Redis was out of memory. With
Elasticache I doubled the size of the instance in place from my phone so I
could go back to sleep. Fixed the key leak in the am and returned to the
original instance size.

~~~
ramraj07
Do you have any details on EB failing?

~~~
senderista
IME EB often gets into unrecoverable bad states that necessitate nuking the
environment and starting over.

~~~
katzgrau
This is the problem. The ways in which it can fail can vary (sometimes you
can't even pull logs from the web interface), and if it fails during a
production deploy you may be left at half or no capacity.

When something inextricably effs up, rebuild the environment and more often
than not, the problem dissapears.

------
helsinkiandrew
> With the Lambda server-less paradigm, you end up with 1 lambda function per
> route

I’m not sure why this is the case. You could host all modules in the same
lambda endpoint /api/v1/* Using the same technology you would use with any
other backend

~~~
damagednoob
Yeah, we use [https://github.com/awslabs/aws-serverless-
express/](https://github.com/awslabs/aws-serverless-express/) which solves the
problem of multiple routes and running an express app locally.

~~~
philliphaydon
Yeah I use ASP.NET Core Lambda.

Article's comments don't seem well researched. Just used tech. Ran into road
block. Blogged. Complained.

------
sputknick
I work in a security office managing AWS infrastructure resources across a
large org and three of these services work great for us:

CloudFormation: this is an excellent resource when you need to tell teams
across our org how you want them to set up resources. Rather than have
numerous lengthy meetings where we tell them what to do, we just give them the
CF template, super simple, and we guarantee everyone has the same setup.

Kinesis: works great for ingesting the data we had them set up resources for
with the above mentioned CF script. I can't speak to the Java dependency the
author mentioned. Not an issue for us. YMMV.

Lambda: Also works great with the CF template setup mentioned above. Super
cheap to use. Maybe the difference between our implementation and the authors
is the frequency and trigger used. Our lambda functions are all time based and
run once a day, or maybe a few times a day. Super reliable, super easy, super
cheap.

I think the overall thesis here is that the usefulness of AWS services depends
on what you are using them for.

~~~
dlhavema
That's a great use case for lambda. Small self contained scripts/jobs that
don't charge you all day to be available once.

------
EdwardDiego
> Furthermore, in order to have multiple workers, Kinesis Streams require you
> to use multiple shards. Each worker will make claim to a shard.

It helps if you think of Kinesis as AWS Kafkaesque rather than a message
queue, because then shards = partitions and how you work with a Kinesis stream
makes a lot of sense.

Multiple concurrent consumers? You're going to need shards/partitions.

Now, question is, as a replacement for homerolled Kafka in EC2, or AWS's
Managed Kafka, is it cheaper?

So far, magic 8-ball says "benchmarking for your given use case needed". So
far my experiments for our workload and patterns say - maybe, but more
investigation is needed. I plan to role out a side-by-side Kafka and Kinesis
experiment on a given topic and ascertain the costs.

Although ultimately, like any distributed messaging system, you end up
engineering to the foibles of the system - in Kinesis' case, it's the fact
that it rounds each sent message (which could be a batch of records, or a
single record) up to the nearest 5KB for billing that you have to engineer
for.

~~~
jpgvm
Kinesis is just a very very poor Kafka.

They wanted to have a Kafka but they also had this silly idea that everything
needed to be HTTP so they built a poor clone with a HTTP interface.

Unless forced into it by some other system there is never any reason to
consider Kinesis over Kafka or Pulsar.

~~~
enitihas
What do you think are the major advantages of Kafka over Kinesis?

~~~
jpgvm
It's actually awfully simple. Take everything Kinesis does and Kafka does it
better. Yes, everything.

Then ontop of that add Consumer Groups which basically deal with the issues in
the OP w.r.t consuming a topic from multiple processes along with providing
administrative APIs to reset application offsets, inspect lag etc.

Also a bunch of extra features like transactions etc but if you are comparing
on the basis of Kinesis like features they aren't likely to matter as much as
the core functionality - which is where Kinesis really gets destroyed anyway.

Normally I wouldn't take such an absolutist position but when it comes to
Kinesis let me repeat in no uncertain terms.

Never use Kinesis.

~~~
saberience
You're missing something major. I have experience using Kafka and Kinesis. My
curreny company only uses Kinesis and zero Kafka. Why? Because Kafka takes
significant knowledge, experience, time to setup, understand, and support. And
if it fucks up and you don't have one or more "Kafka experts" on-call, well,
you're screwed.

Kinesis on the other hand, just works. Yes, it just works. I don't have to
have a Kinesis expert on hand, I don't have to configure clusters myself or
write Ansible/Puppet, etc. I have a few basic lines of Terraform to create my
Kinesis streams and I push data to them and we got it working in minutes and
we've had no issues.

Contract this to my previous job where we literally had to hire multiple Kafka
experts at high salaries to maintain our Kafka clusters.

This is why you only use Kafka if you ABSOLUTELY NEED it.

~~~
jpgvm
I am about 90% the way ridding a larger company of Kinesis. You might not have
needed a Kafka expert but instead you had to create (likely poorly
implemented) frameworks for consuming from Kinesis (when you could have just
used Consumer Groups). Also Kinesis is insanely expensive, to the point that
if your throughput is anything but trivial (in which case it doesn't matter
which option you use, both will -just work-) that is just becomes untennable.

I actually did miss something but it's not complexity, you can easily use
hosted Kafka (AWS even provides it with MSK now). It's actually
authentication. Kafka does have authentication but it doesn't easily tie into
AWS or GCP authentication mechanisms without the help of a tool like Vault.

That said.. I don't think that minor benefit is enough to ever justify Kinesis
over Kafka.

~~~
crescentfresh
> Kinesis is insanely expensive

No, it is not. Also, an equivalent cluster of EC2 instances running
zookeeper/Kafka/etc will outprice Kinesis.

~~~
jpgvm
At tiny throughput maybe? Pretty sure you could run Kafka on small ec2
instances and still get great throughput, especially if you don't need much
retention. (reducing retention reduces need for disk I/O as consumers can't
fall far behind causing disks to seek).

I think for any reasonable workload, i.e 10k/s+ and/or throughput over 100mb/s
Kinesis gets dumpstered for price every time.

------
noirbot
I'm confused that the author says about Cloudformation:

> Lack of Drift detection or reconciliation. With lack of drift detection
> comes great uncertainty.

Cloudformation has definitely had Drift Detection since 2018:
[https://aws.amazon.com/blogs/aws/new-cloudformation-drift-
de...](https://aws.amazon.com/blogs/aws/new-cloudformation-drift-detection/)
It's not everything you might want it to be, but it's not like Terraform will
reconcile your drift automatically either, that I know of.

~~~
ReidZB
Drift detection is only implemented for a very small subset of resources:
[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-
cfn-stack-drift-resource-list.html)

Terraform does indeed reconcile drift 'automatically' across all resources, by
which I mean the plan will include changing everything that's drifted back to
the specified configuration. That may not always be desirable, which is why
building a good plan/apply process with approval is important. (Same goes for
CloudFormation, though.)

------
karterk
Can't believe Cloudwatch wasn't mentioned here. Could be such a useful product
that is so messed up that there is probably multiple billion dollar companies
in this space.

~~~
zwily
I’ve heard rumblings from inside AWS that there is internal pressure not to
make CloudWatch too powerful, lest it harm their partners (like Datadog, etc).

I’ve heard this a couple times after complaining to AWS engineers about
CloudWatch shortcomings.

That said, CloudWatch has gotten tons better than a few years ago.

~~~
pentlander
I don't think that's true, internally there was a long term plan to bring
cloudwatch up to par with internal metrics and alarming systems. At the time
when I left Amazon, there were some shims to use internal dashboards and
alarms with cloudwatch data. The eventual goal was for the whole company to
move to cloudwatch instead of maintaining two systems.

~~~
somebodythere
What I don't get was the broad internal push to building on AWS directly,
which was great most of the time, but came with big downgrades in metrics and
alarms if you were to use the AWS equivalent.

------
eraad
I believe the title of the article should be "AWS Services You Should Avoid if
you don't need them or if they are not the right fit for your specific
application, for example, if you need social login you are better off with
other alternatives more fit to the social login use-case; or for example, if
Lambda is not the best fit for containing and deploying my code given my
architectural decisions at the moment of this writing".

~~~
kdidbsbsbd
That is a very long sentence

------
Twirrim
I'm honestly not familiar with most of the services mentioned from a user
perspective, the AWS service I worked on didn't consume them at the time, and
I work for another cloud platform now.

That said, this point gave me particular pause, because it brought in to
question the other assertions:

> The application collected small json records, and stuffed them into Kinesis
> with the python boto3 api . On the other side, worker process running inside
> EC2/ECS were pulling these records with boto3 and processing them. We then
> discovered that retrieving records out of Kinesis Streams when you have
> multiple worker is non-trivial.

Yeah.. because that's really not what Kinesis is designed for. They even,
rightly, point out that SQS is a better fit for that purpose. That hardly
makes it a service to avoid.

~~~
senderista
That was the most annoying part of the article. A streaming data platform is
not a task queue. Why do you think they built Kinesis in the first place when
SQS had been around for years?

------
CSMastermind
I wholeheartedly agree with the Cloudformation take.

Terraform is better in almost every way. If you use Cloudformation you'll end
up writing a bunch of bash script wrappers or similar around it to make it
actually do what you want.

Everyone who's tried both at scale has said the same thing.

~~~
boredgamer2
Terraform is better in almost every way.

I disagree. I spent time in Terraform a few years ago working with a client
and Terraform had the ability to create but not tear down resources for some
services. I was shocked -- check out the Github issues history. I ended up
writing a "bunch of bash script wrappers or similar around it".

~~~
noirbot
Yea, this is exactly what I was talking about. It's not to say that
Cloudformtation can't run into the same thing - deleting S3 buckets is
difficult in any situation - but there's a lot of things Terraform
doesn't/won't do, and often you're left with just making your own second layer
of automation to work around.

------
Sytten
Used Kinesis in an ETL pipeline and we ended up writing our own custom client
for it because we didn't want to use the Java one. The product itself is not
too horrible, but the fact that they refuse to create native clients for other
languages makes it a no go for any company that is not JVM based. A managed
Kafka on GCP or directly a kafka on Kubernetes (there is a controller for that
if you pay confluence that makes the whole thing painless) is much better.

~~~
calcifer
There are excellent community KCLs for other languages, like Twitch's kinsumer
[1] for Go. We've been using it in production with a high througput stream for
almost a year now without any issues.

[1]
[https://github.com/twitchscience/kinsumer](https://github.com/twitchscience/kinsumer)

------
2ion
While CloudFormation is annoying to use without extra tooling (like Sceptre --
makes management of dependencies between outputs and inputs go away easily,
and it's extensible with plain Python), Terraform is horrible in its own ways.
The AWS provider has many consistency issues, even glaring ones like not
detecting that IAM policy documents have not changed when using lists in
actions=[] and resources=[], and after having tried it, it makes me
uncomfortable managing large landscapes with it.

Terraform makes mostly sense -- because there is good tooling around
CloudFormation, and CFN more often than not does the well enough -- when you
find out that CloudFormation is like Terraform just a driver for AWS APIs,
likely developed by a dedicated team, and as such features of the underlying
APIs are often not exposed at release time in CloudFormation. I noticed this
especially when experimenting with EKS: There's eksctl, which integrates with
CloudFormation (generates some stacks) but is utterly useless for integration
because you can't import outputs or exports, so you have to hardcode all your
SG IDs, VPC IDs, Subnet IDs if you want to integrate with existing
infrastructure
([https://github.com/weaveworks/eksctl/issues/1141](https://github.com/weaveworks/eksctl/issues/1141)).
Dealbreaker, waste of time, disappointing for an "official" CLI. Next, there's
pure CloudFormation -- but no luck for you, to this day AWS doesn't support
EKS endpoint access control settings through CloudFormation
([https://github.com/aws/containers-
roadmap/issues/242](https://github.com/aws/containers-roadmap/issues/242) \-
dealbraker, if you need that and are allergic against public control endpoints
in your infrastructure. Looking at Terraform -- it supports integration with
existing CloudFormation infrastructure and can access CFN exports, it supports
all the EKS settings you'd want, it offers a consistent interface to these
features, so you got something to use, short of rolling your own.

Mind you, CloudFormation is extensible using Custom resources, but hacking
around CloudFormation is likely not worth your time and something Amazon
should do. Anyway, CloudFormation is likely one of the most tested and well-
working parts of AWS, so I'd prefer it over third-party state management
unless I find a very good reason to do so.

~~~
scarface74
_so you have to hardcode all your SG IDs, VPC IDs, Subnet IDs if you want to
integrate with existing infrastructure
([https://github.com/weaveworks/eksctl/issues/1141](https://github.com/weaveworks/eksctl/issues/1141)).
Dealbreaker, waste of time, disappointing for an "official" CLI_

You can reference Parameter Store and Secret Manager entries in CF so you
don’t have to hardcore values.

------
raverbashing
"What’s the problem (with Elasticache)? It is expensive, and no one notices
for months."

So you just click on stuff without knowing how much it costs? Really?

> Secondly that cache.* prefix means this instance costs $0.216/hr instead of
> $0.126/hr, a 71.4% premium. Then you might think you need one for dev, qa,
> and prod

Ok and you pick a cache of the same size for dev/qa/prod? Again, really?

> But, not all data/records/events should go into Kinesis. It is not a general
> purpose enterprise event bus or queue.

Yes, there's SQS for that. But it seems another case of clicky-go-lucky. "Hey
it says queue here so we'll just use this right?!"

> Lambdas are great for the following tasks:

Yes, agreed. I would generalize it as: it is good for specific small tasks,
especially when plugging into the rest of the AWS infrastructure (reacting
to/from SQS, S3 events, etc)

> Lambda is horrible for: > A replacement for REST API endpoints.

 _Sigh_

General advice: if something feels weirdly hard, then you're probably doing it
wrong.

------
acdha
> the new YAML CF is better than the old JSON CF, but it’s still tough to read
> long complex stacks.

These days I would recommend using CDK if you can’t switch to Terraform but I
would never under any circumstances recommend YAML since the odds approach
certainty that the magic in the parser will cause a problem. Beyond the usual
confusion around typing I’ve seen things like significant whitespace breaking
functionality[1] and the shorthand types in a few cases make diffs messier. If
you find an example in YAML, it takes a second to run it through cfn—flip and
you’ll never have to deal with any of that.

For anyone using CF seriously, I highly recommend the cfn-python-lint tool and
associated editor support. It will catch many of these cases before you burn a
lengthy update cycle.

1\. Fun fact: AWS’ own security alert CF stack fails the Security Hub CIS
scanner because it adds extra white space the metric filters.

------
cddotdotslash
> With the Lambda server-less paradigm, you end up with 1 lambda function per
> route.

Only if you use a framework that splits it up that way (which I agree is
horrible). But nothing about Lambda _requires_ this architecture. In fact, API
Gateway has a proxy mode that allows you to serve all requests from a single
Lambda.

------
philwelch
To my surprise this is mostly solid advice despite the clickbaity title.

I've never used Cognito or ElasticCache, but for the other three
(CloudFormation, Kinesis, and Lambda), I would agree with what he's actually
saying, which is to use these services judiciously.

CloudFormation has some advantages over Terraform, but having used both,
Terraform is much more usable for long-lived static resources. I have yet to
try Pulumi, and I don't see any reason why I should to be honest. However,
CloudFormation is required for automated build and deployment of Lambda
functions--both Serverless Framework and AWS SAM use CloudFormation. I've also
used CloudFormation StackSets to build out infrastructure for multi-account
governance. In normal use though, Terraform is typically more usable, though
both solutions can get pretty hairy.

By "Kinesis" the author seems to be referring to Kinesis Streams. In fairness
I haven't built anything where Kinesis Streams would be a better use case than
SNS/SQS, but that's just as much a statement about the projects I have
happened to work on than it is about Kinesis Streams. I have used Kinesis
Firehose, which provides a very scalable mechanism for, "multiple clients are
intermittently spitting out a massive volume of data points, all of which need
to be faithfully logged into S3 somewhere".

And finally, we come to Lambda. Lambda is a good fit for use cases where
either you need to run some code on an intermittent basis or when you want to
prototype faster and don't want to mess around with provisioning and deploying
to servers. Lambda is a great place for little pieces of glue code that get
triggered on a x-minute cron or based on a CloudWatch event or from SNS or
SQS. It's good for serving an HTTP API that gets less than one request per
second. These use cases are extremely common and I've encountered them a lot
more than the author seems to, though again maybe that's just me. But for
anything high traffic, where the Lambda container is just going to stay hot,
just deploy it to EC2 (perhaps with some container magic in the middle), it's
cheaper in the long run.

~~~
x3n0ph3n3
My team has invested a sizable amount of code to manage CloudFormation. At
this point, we have a rather mature interface for dealing with CloudFormation,
including:

\- defining dependencies between stacks

\- taking outputs from stacks and feeding them in as parameters to other
stacks (not using that awful Import/Export Value crap they implemented)

\- deploying as many stacks in parallel as possible and waiting for them to
complete before deploying dependent stacks

\- dealing with common failures and rollbacks, including handling known
"continue update rollback" steps with predefined resources to skip

\- pre/post stack create/update/delete actions to make API calls and perform
other actions outside of CloudFormation

We basically built the missing pieces of CloudFormation ourselves and have
managed to keep CloudFormation holding the definitive state rather than
managing it ourselves (or paying Terraform to do it). We have about 500 stacks
for a single deployment of our product.

We also implemented a large number of Transform Macro lambdas to composable
templates _much_ easier.

There's a lot you can do with CloudFormation, but it takes some investment.

A few things about it that really drive me nuts, though:

\- Lagging support for new features/resources

\- Parameter count and parameter size limits

\- Certain bugs with some resource types that are slow to get fixed. (Redshift
cluster password management and issues with elasticsearch resizing triggering
blue/green deployment come to mind.

\- No ability to modify timeouts for some actions. For example, the timeout
for a CustomResource is fixed and cannot be tuned -- if your Lambda never
responds, CloudFormation will hang up to 2 hours. We wrote our own Lambda
wrapper just to guarantee a response if an unexpected failure occurs

------
smitty1e
> With the Lambda server-less paradigm, you end up with 1 lambda function per
> route.

You do? Why can't a function be written to handle multiple routes?

~~~
floydnoel
It certainly can be done. The author seems to be uninformed on the topic he is
writing about.

~~~
smitty1e
To the author's point, fragmentation is a threat if there is no architectural
guide.

------
karmakaze
This would be great as a Markdown repo of "When to avoid each AWS Service."

Many more than the listed services have serious usage limitations or gotchas
and the listed ones have more limitations. Workarounds would also be useful to
know.

~~~
dehrmann
There's also ones that aren't in active development like SimpleDb. It's not
that it doesn't work, but you probably don't want to build anything new with
it.

~~~
karmakaze
I don't get this attitude when applied to small things. Can't it just be
considered complete and not in need of enhancements? If it's not supported to
the point of not fixing security issues that's completely valid.

~~~
duskwuff
SimpleDB has some significant limitations which don't exist in its successor,
DynamoDB. For example, SimpleDB tables are limited to a size of 10 GB; no such
limitation exists in DynamoDB.

~~~
karmakaze
Well stated. It's very clear to say it has a 10 GB limit rather than a vague
"don't use this". SimpleDB and DynamoDB are at opposite ends of the scaling
spectrum.

------
zxcvbn4038
Someone’s shotgun of frustration post, sounds like the author is mad at
everything in AWS. Lambda has its uses - I definitively wouldn’t follow AWS’s
SAM model and I wouldn’t try to do serve my critical web site from it - but
lambda does have a convenience factor.

Cloudformation isn’t perfect but it is well integrated within AWS and you can
get support for it around the clock. Hashicorp quoted me $13,000 just for
support for Terraform. AWS support isn’t cheap either but covers everything in
AWS and their support is fantastic - like IBM in their heyday.

------
scoot_718
Athena is actively worse than hosting your own Hive on EC2. It'll timeout
queries and give you no results beyond charging you. Given the variable
performance of S3, that's terrible.

~~~
aeyes
Wait? Why are you comparing Athena to Hive? Athena is basically managed
Presto.

------
bshanks
Summary of the article's recommendations:

Don't use: \- Cognito (authentication). Because: on mobile social login it
isn't native. Instead: use Auth0, OneLogin, Okta or roll your own

\- Cloudformation (programmatically configure AWS). Because: various
complexities. Instead: use Terraform.

\- ElastiCache (managed Redis). Because: expensive. Instead: run Docker Redis
in EC2.

\- KINESIS (queue), as a general-purpose data queue. Good for: streaming data
such as video processing/uploading. Bad for: generic data queue, because its's
difficult to route each event in the queue to one of multiple workers. Kinesis
is meant for every listener to be assured to get the entire stream. Instead:
SNS/SQS (SQS FIFO) or a queuing framework that sits on top of Redis or
traditional databases.

\- Lambda (server-less), to implement a REST API. Good for:
Serving/Redirecting requests to CloudFront, also reacting to events from SNS
or SQS by running small asynchronous tasks. Bad for: replacement for REST API
endpoints, because it's too hard to work with a zillion lambdas. Instead: use
a regular web framework (or, other commentators say, route all requests to one
lambda and then do more routing within than lambda)

I do not agree or disagree, I'm just summarizing; although I did add in some
information from other comments here, too.

------
peterwwillis
You should use Cognito as a quick way to provide SAML/OAuth to a lot of apps,
but it's not going to solve all your authentication woes. Nothing does (not
even Auth0).

You should use CloudFormation if it's there and you're just trying to stand
something up quickly. Shell scripts with AWS CLI works too. For more robust
long term stuff, use Terraform, Pulumi, or custom Boto3 code. The point is to
just start using the Infrastructure as Code pattern early, not to make it
perfect. Terraform is surprisingly painful after a while, but it's easier to
standardize your organization on large-scale. There's no great solutions in
this space.

You should absolutely throw money at ElastiCache to quickly scale up and down
a cache. If you're not a very experienced sysadmin (and fuck the entire tech
industry for making that term a dirty word - _if you are not experienced at
administrating systems_ ) you'll be spending unnecessary time and energy to
stand up and maintain it. Your caching should ideally just be in your service
and scaling your services themselves should be sufficient, but whatever, Redis
wasn't invented for people who write good code.

 _> But, not all data/records/events should go into Kinesis_

Not all of anything should go into anything. Use it if it's convenient.

Lambda is very useful for batch jobs and CloudWatch-triggered maintenance
tasks. I wouldn't rely on it for anything more than that; I would rather just
deploy the same code to a Fargate Spot Instance and not deal with all of
Lambda's bullshit.

------
typenil
With regards to Cognito, I'd recommend reading through this GitHub issue
before committing to it:

[https://github.com/aws-amplify/amplify-
js/issues/3495](https://github.com/aws-amplify/amplify-js/issues/3495)

While I haven't personally had the opportunity to run into these issues, the
feedback there shows a serious lack of ownership that I've never encountered
elsewhere with AWS.

------
serguzest
Medium doesn't let me read this unless I pay. You should avoid medium blog
writing too.

~~~
mdaniel
While I agree that medium should die in a fire, it has been my experience that
someone has usually submitted the URL to archive.is and that is true in this
case, too: [https://archive.is/hz1RG](https://archive.is/hz1RG)

You may also enjoy trying an incognito window in the future

------
WatchDog
Regarding using lambda for a REST API, just set up API gateway endpoint in
proxy mode, and run all your endpoints from one lambda, will be much simpler,
and if you use a decent framework, it will be really simple to convert your
app into a regular daemon process if you ever decide to move off lambda.

Regarding cloudformation, I view manually writing cloudformation json/yaml as
an anti-pattern. I would recommend looking at using CDK, which is a framework
for writing your cloudformation stacks as actual real code, not markup. It's a
great tool, and I find it enormously more productive and expressive than
terraform.

CDK allows you to write CDK apps in a variety of languages, but I would
recommend just using typescript, since that's what its written in. I tried the
java bindings, but it was pretty clunky.

For a non-AWS specific CDK like experience, you could also look at using
pulumi, but I haven't really used it much myself.

Trying to define infrastructure purely by declarative markup based languages
is such a waste of time.

~~~
mvanbaak
Even if you dont want to use the proxy route, you can point multiple routes to
the same lambda function, and every ok-ish routing module/framework will know
what code path to follow.

------
samstokes
This makes some good points about misuses of these AWS services, but the title
is misleading. The article is actually more like "tempting but inadvisable use
cases for AWS services".

My employer uses three of these heavily (ElastiCache, Kinesis and Lambda) and
we get quite a bit of leverage out of them.

ElastiCache in particular surprised me. At first glance I mistook it for a
transparent (and expensive) wrapper around sticking Redis on an EC2 instance,
but if your usage is heavy enough to need multi-node clusters (e.g. read
replicas or full Redis Cluster), its orchestration features are pretty useful.
We can resize instances, fail over to a replica, and reshard clusters, with
zero downtime, by clicking a button (or a one-line Terraform change). And
never having to install security patches is nice too.

It certainly is expensive, though. (But if you're not willing to pay a premium
for managed infra, what are you doing on AWS in the first place?)

------
Scaevus
Just like the Redis replacement, AWS's MongoDB replacement of DocumentDB comes
us when you search for Mongo, but it is not an actual replacement even though
the name indicates "compatibility."

As a rule of thumb, we always try to avoid such proprietary services for
potential vendor lock-in, whether AWS or not.

------
mattacular
The CloudFormation one is pretty accurate. Relying on nested stacks to "share"
templating pieces: never again.

~~~
gonzo41
And that's exactly the feeling I had, but hey we're in prod now, so we'll
circle back in 5 years to see what we do next.

~~~
mattacular
Don't fret! You still have to look forward to all the weird service limits
you'll hit both directly and indirectly using CFN to maintain and extend a
prod infrastructure YoY

------
abrookewood
I'd add Route 53 Resolver [0]. $90 a month for a few DNS queries seems
completely mispriced.
[https://aws.amazon.com/route53/pricing/](https://aws.amazon.com/route53/pricing/)

------
thanksforfish
> Cloudformation

The approach CFN used made sense when there was no alternative, but
Terraform's approach is so much better. I wonder how AWS approaches this space
in the future. CFN has great vendor lock-in, and needs to remain supported for
any customers who are locked in, but its harmful for any new development to
use CFN.

I get that AWS likes having infrastructure framework level lock-in, but they
need to do better.

Does anyone have a feel for how hard AWS support, account reps, or training
push CFN versus Terraform? I've been out of the AWS cloud for a while.

~~~
trill123
Support doesn't officially support third-party products, and support engineers
may or may not know Terraform. Engineers definitely aren't trained on
Terraform, and your mileage will vary a lot depending on the specific engineer
and their background. It really depends on whether they've run across it in
other jobs, personal projects, etc.

Customer obsession is a big deal for support engineers. So if they can,
they'll try to help solve a Terraform issue. But it'll be on a personal, best-
effort basis.

For such a mud-slinging article, I'm surprised the author didn't complain
about the most pressing issue (IMO) with CloudFormation: There have been a few
instances where thing in AWS changed and Terraform was updated more quickly
than CloudFormation was. Happens less and less now though.

~~~
exidy
> Support doesn't officially support third-party products

This is not entirely correct. "AWS Support Business and Enterprise levels
include limited support for common operating systems and common application
stack components" as documented here:
[https://aws.amazon.com/premiumsupport/faqs/](https://aws.amazon.com/premiumsupport/faqs/)
under "Third-party software"

------
ciguy
Cloudformation is spot on. There is simply no excuse in 2020 to use it when
cloud agnostic Terraform exists.

Elasticache is expensive and the analysis is generally correct on that,
however if you do actually need a high capacity rock solid Redis cluster and
you can afford it, it does the job. Paying someone to setup and scale a
cluster of that magnitude would probably cost a year's worth of usage. Of
course if you don't actually need to scale it yet then just use the
suggestions in the article to run it yourself.

~~~
scarface74
Terraform is not “cloud agnostic”. Every provisioner is specific to the
platform.

~~~
ciguy
Perhaps bad terminology on my part but I would argue that Terraform IS cloud
agnostic, it is NOT cloud portable.

You can use Terraform with any cloud. You cannot seamlessly move your AWS
configuration to Google Cloud but you can still write Google Cloud
configuration with the same paradigm as your AWS configuration.

~~~
jen20
This is exactly the idea of Terraform - the workflow is portable and agnostic,
but the full fidelity of each individual resource provider API is exposed.

~~~
jatone
which is fantastic. =)

------
dnprock
There's another one that I think qualifies for this list: Elastic Container
Service. It's a half baked Docker service. It seems like it's abandoned now
for Kubernetes.

~~~
shanth
Abandoned as in? I'm considering ECS for some projects. Will EKS a better
sword than ECS?

~~~
vegardx
I don't really agree that it's been abandoned, nothing suggest that this is
true. They're not known for just abandoning established services, there's a
public roadmap[0] that you can follow. Like all services there are pros and
cons. Up until very recently EKS lacked more or less everything ECS has in
tight integration with other AWS services, so there's that.

[0] [https://github.com/aws/containers-
roadmap/projects/1](https://github.com/aws/containers-roadmap/projects/1)

------
vagab0nd
I have mixed feelings about Cognito.

I love the simplicity.

I love the 3rd party integration that comes with it.

I love the fact that I don't have to manage user credentials and everything
else by myself.

However,

I hate that the UI is not polished, and there's very little customization
allowed. As an example, from the sign in/up page, you can't add a link to go
back to the home page.

I hate that if user signs up via Google, and later on tries to reset password
via the Google email, Cognito silently drops the request.

I hate the js library. Hard to use and documentation is not great.

------
buzzy_hacker
AWS CDK can generate CloudFormation templates, it’s pretty cool:

[https://aws.amazon.com/cdk/](https://aws.amazon.com/cdk/)

------
virmundi
You can use one lambda to handle multiple API gateway requests. Route to ANY.
Then have a bridge that connects to your favorite language router. We do this
all the time with NestJS. The down side is you burn some cash when someone
calls a 404 resource since your NestJS has to say that it’s missing and that
requires at least 100ms of lambda time. You could probably get around this by
generating a route maker off the NestJS controller files.

------
catchmeifyoucan
You can definitely use one Lambda function to serve all of the endpoints that
the author mentions. You need a wildcard route in API gateway, and can easily
use koa-serverless or express-serverless to get the route information when any
request is received. That way you only need one function for your APIs. Even
with a regular api not on lambda, you’ll still have to be aware of db
connections.

------
paxys
I have no experience with Cognito and the CloudFormation point I completely
agree with. The rest of them don't make sense though. ElasticCache is
expensive but offers a ton of features that a single Redis process on an EC2
instance will not. And while Kinesis and Lambda are bad for the use cases
called out in the article, they each fit their niche pretty well.

------
timmit
I sort of disagree with the EC opinion.

you pay what you get.

AWS's service is always more expensive than the service you built, but at the
same time more realible too

------
jtdev
For Python based Lambda REST API development, check out Chalice:
[https://github.com/aws/chalice/](https://github.com/aws/chalice/)

Very Flask like and includes CLI components to automate deployment. Even
includes simple decorator based event mapping for S3, SQS, scheduled tasks,
etc.

------
alexnewman
I agree. My personal list of stuff I like using at Amazon 1\. S3 +cloudfront
(but I still use cloudflare for ddos) 2\. Ec2 3\. Pg aurora 4\. Sqs is a few
cases, but generally i use redis for queueing 5\. Lambda for tiny
notifications usually to sqs if s3 gets dinged

Unlike the author I'm not afraid of lambda but generally prefer containers for
apps

------
marcrosoft
The lambda part it’s just wrong. You can use apigateway as a proxy to avoid
the mess of one lambda per route.

------
toredash
Author misses one important point with Redis: you don't pay for traffic. If
you host it yourself, you pay for traffic both way if traffic crosses AZs.
Dynamic scaling, no downtime upgrades,æ. It works really well.

I do hope they will get tiered RIs for ElasticCache like they have for RDS.

------
thanksforfish
> You dig into CLOUD WATCH logs. We have three sets of logs all intermixed
> together. This is unreadable.

I'm not following this. You don't read logs one by one. You filter by source,
error code, time range, etc. Its been a while, but I'm also pretty sure CW has
log groups.

~~~
trill123
> Its been a while, but I'm also pretty sure CW has log groups.

This. If you have three sets of logs intermixed together, things have been
architected incorrectly.

------
acd
Also avoid EFS and please use S3 instead. Elastic Filesystem which is not that
"elastic".

------
anthony_barker
Hmm. 1) their VPN solution is overly expensive 2) Mongodb document database
similar 3) Lambda I want to use but other than the sns/sqs connected tool I
agree as an api endpoint google's offering is better 4) Cognito horrible
docs...

------
StreamBright
>> With the Lambda server-less paradigm, you end up with 1 lambda function per
route.

Factually incorrect. It takes as many routes as you want. I mostly ok what he
is saying but some of his claims are just factually wrong.

------
impowski
One time a hosted an ElasticCache for testing purposes for Bull library for a
couple of days and then I forgot about it for a month but I wasn't even using
it. Got the bill for 1500$ :)

------
pinkskip
We use the above mentioned services and I don't agree with the assessment as
we are cloud native and use lambda heavily and we think its an amazing service
to use to move faster!

------
kbumsik
> CloudFormation

I lack experience in AWS or clouds but I wonder what is the alternative to CF
if I should avoid it? I can imagine alternatives to the rest but I cannot come
up with one to CF.

~~~
acd
Use Terraform instead or create infrastructure with Pulumi using conventional
programming languages such as Typescript, Javascript and Python.

Terraform [https://www.terraform.io/](https://www.terraform.io/)

Pulumi [https://www.pulumi.com/](https://www.pulumi.com/)

Hand written Cloudformation is on Hold on thoughtworks technology radar.
[https://www.thoughtworks.com/radar/tools/handwritten-
cloudfo...](https://www.thoughtworks.com/radar/tools/handwritten-
cloudformation)

~~~
kbumsik
> thoughtworks technology radar

I've never heard of this magazine but it like a very useful curated list.
Thanks for mentioning this!

------
rantwasp
yeah no.

cloudformation is awesome and it works as advertised. I have literally not
seen it fail catastrophically ever since it came out. it just works.

terraform? or terrafail as it’s known. it’s a disaster. it cannot keep track
of the resources it creates. when in trouble, it throws its hands up in the
air and you’re on your own. do yourself a favor and don’t learn the hard way -
in production - that terraform cannot take your infra from point A to point B
(and rollback in case something went wrong).

~~~
simlevesque
Yeah I never got the CloudFormt1ion hate. I write them for a living without
any issue.

~~~
rantwasp
it’s not cool to use it. why use a service that’s written, ran and maintained
by a dedicated team that also builds all the other services you use? (if in
AWS). they don’t know what they’re doing, right?

better use some randotool and talk big about Iaac when in reality the only
people that I have ran across and loved Terraform either 1) used it on a
really small scale or 2) don’t really have a lot of experience with the cloud.

the only reason to not use Cloudformation is that you’re not using AWs. Use
whatever your cloud provides.

For everyone that has objections, an appeal to authority: I was building stuff
on the cloud before you knew the cloud was a thing. I worked with AWS, GCP,
Azure and [sad face] OpenStack extensively.

------
Yhippa
> With Cognito, they are presented with an embedded WebView, prompting the
> user to log into Facebook again in the embedded WebView.

Off-topic, but this is an antipattern right?

------
caogecym
I’ve been using Okta for authentication in my SaaS app, the doc and support
experience had been great for me! Free 1000 MAU. Haven’t tried Cognito or
Auth0.

------
WilliamEdward
I know it still makes sense but 'amazon web services services' is funny

~~~
oars
Just like ATM machines.

~~~
lucb1e
Yep
[https://en.wikipedia.org/wiki/Pleonasm#Other_forms](https://en.wikipedia.org/wiki/Pleonasm#Other_forms)

------
dom96
Am I the only one that finds AWS' pricing structure as a whole too risky to
use? Why use a service that can charge me a lot of money without being up
front about it?

------
blickentwapft
Where’s dynamodb?

~~~
goostavos
DynamoDB is fine __as long as it fits your use case. __

The trouble is that without a time machine, it 's very tough to realize that
your use case isn't the one DynamoDB is good at. The docs and Best Practices
certainly suggest that anything is possible.

Do you need God Tier Scale (and understand enough about DDB to achieve it (and
have the $$$))? DynamoDB is awesome! Otherwise... maybe consider a few other
options...

------
marta_morena_24
Deep analysis of the situation! Guys, avoid those 5 services haha...

------
swyx
anyone have a non paywalled link? hate Medium

~~~
mdaniel
[https://archive.is/hz1RG](https://archive.is/hz1RG)

------
gerbilly
I can't say I liked one of his proposed alternatives: Terraform

I tried it and found the experience to be far inferior to Ansible, especially
if you provision actual Linux instances with it.

Terraform was always getting stuck in an inconsistent state, and trying to
roll back, which doesn't work well for any type of stateful resource.

I guess Terraform may be ok for provisioning resources which are stateless and
interchangeable, say security groups and other simple services.

I can't say I enjoyed using it much, though the rest of Hashicorp's offerings,
Nomad, Consul etc are really great.

~~~
allanmacgregor
Well now you are comparing apples to oranges, terraform main focus is managing
and provisioning infrastructure while ansible is more commonly used to do the
actual instance provisioning and configuration.

In fact, Terraform + Ansible is a fairly common and powerful stack.

