Not a great article. Most of the content is far better explained in the AWS docs.
To summarize their best practices:
* don't put creds in templates
* reference other parameters outside of the template, e.g. in Systems Manager Parameter Store
* make your code readable
* add comments
* check your code
Except for the second, all of those are fine practices for maintaining any code.
The one best practice that's actually relevant to CF is to store parameters in SSM. Yet, they mention that drift is a problem with CloudFormation. So... why do you want to store parameters to your stack in a service that allows them to change independently of the stack?
Maybe there's an argument for doing this, but they don't explain it. It's just a "best practice."
They don't mention a major use case for CF, namely CI/CD. The article finishes up with a pitch for their template designer.
Man, I don't miss CloudFormation. If you want to do anything interesting at all (like being able to spin up turn-key ad-hoc stacks for developers that are mostly the same as your main environments), then you have to parameterize everything, and passing data around CloudFormation templates is just awful--what you want is functions and data structures but CloudFormation gives you "nested-stacks" and you have to encode most of your data structures as strings and "parse them out" with the YAML builtin functions that they provide (e.g., `{"Fn::Split": "foo,bar,baz"}` will evaluate to `["foo", "bar", "baz"]`, but good luck if you want a list of objects). Each stack also has a limit on the number of parameters you can pass, and since you can't really pass struct-like data (e.g., `"Foo": {"bar": "baz", "qux": "..."}`) you end up flattening that struct into N parameters (e.g., `"FooBar": "Baz", "FooQux": "..."`). You can try to plumb things through SSM or SecretsManager, but each stack also limits the number of parameters you can pass there as well.
Basically, CloudFormation is like the shittiest programming language you've ever encountered. People will argue that it's supposed to be simple because it's just YAML, but that's bogus--we clearly need to be able to do complex things in this space or else CF wouldn't provide these hacky functions. We pretty clearly need some sort of expression language if not something more robust, since that's the direction all of these "it's just YAML!" tools are going (CloudFormation, Terraform, etc). CloudFormation might actually make a decent "assembly language" for an infrastructure-as-code backend (although extending CloudFormation for third party services was still far too difficult last I checked--basically you had to run your own lambdas) that some higher-level tools might generate, but it's a mess for anything that isn't a toy.
AWS solutions architect here! (Opinions are my own and not necessarily those of my employer.)
Many of us now believe that CloudFormation is best described as the assembly language of AWS infrastructure: you can hand-code it, but it's challenging and verbose, and there's a lot of detail you have to get involved in.
CDK is the high-level solution to this challenge. With CDK you describe your infrastructure in terms of high-level constructs such as classes. Your stacks are described as applications instead of YAML templates. You can use many different programming languages including TypeScript and Python, and code is reusable. You can even vendor custom construct libraries to share within your org, or openly via npm.
I've become a convert since CDK came out - it's been ages since I've handwritten CloudFormation templates.
Yeah, I haven't given CDK a good shot yet. I did write something like it (for Python) before CDK existed, and it was very nice. I'd like to play with CDK, but I'm no longer working on AWS systems but rather GCP/Terraform. Best of luck to the CDK folks!
Yeah JSON/YAML is (mostly) great for data that needs to be parseable by machines and humans but trying to make a language out of it is horrible.
Raw HCL2/Terraform is marginally better IMO with being able to pass around more complex structs and function calls that don't have to be expressed as JSON dicts, but still remaining declarative to be able to create a dependency graph internally.
All of this is why CDK now exists to be able to express these things as TypeScript or Python and spit out the corresponding CFN or TF.
It's also worth noting that AWS added support for additional providers a while back so the CustomResource+Lambda you're describing isn't the only way to interact with non-AWS APIs.
I'm actually curious what you're using - you've shit all over a bunch of things without providing any real alternative.
SAM is limited and I didn't care for it. I believe CDK is the way to go; I just haven't had much chance to play around with it yet. I suspect it doesn't solve the problem that extensions are lambdas, but it's a welcome development. I believe Terraform now has a CDK as well.
I wrote a prototype thing that was like Troposphere but better (everything was generated from CloudFormation spec JSON files, no stitching together things by string IDs, etc). Also unlike CDK, there was no inheritance (I haven't used CDK that much, but inheritance-based systems seem to be universally awful, so I'm skeptical). It can be pretty nice.
Imagine CF being Terraform + Terraform Cloud (only free!) - but more reliable and having real changesets with more predictable behavior and the state being the true AWS state, not some projection of it within Terraform.
I've had too many CF stacks get stuck in states that don't allow for rolling forward or backwards to buy this. You have to purchase a premium support plan to get access to the AWS personnel who can help you out.
CF does actually kind of try to behave transactionally, which is interesting--you don't often get stuck between states like you do with Terraform, but I rarely have had much of a problem getting Terraform out of one of these states either.
This isn't to speak about the abysmal expressiveness of CloudFormation. Reuse is a joke--I'm tempted to call it a programming language in which you encode the AST manually as YAML/JSON, but that would be overly generous because any programming language AST allows you to express the concept of a function that takes complex arguments (e.g., an object, list, or list of objects).
Further still, if you want to extend CloudFormation to support third party services (e.g., if you want to create a user pool in your identity management provider for your new stack, or otherwise manipulate non-AWS APIs), you have to write and run your own lambdas--which each require their own infra-as-code--which just takes a lot of effort (not to mention how hard it is to write these correctly) and so you end up compromising with a half-manual workflow, missing out on much of the promise of infra-as-code.
I don't love Terraform, mind you. I think there's a Better Way out there somewhere (maybe it's Pulumi or CDK, I'm not sure), but CloudFormation is brutal.
This is... quite the assertion. Terraform has historically been _far_ more reliable than CloudFormation, and perfectly capable of getting itself into unrecoverable states. Furthermore, the behaviour has _never_ been predictable - CF only even grew the ability to preview changes relatively recently.
(Disclaimer: I worked on Terraform at HashiCorp, and on other provisioning tools since).
For a person who uses Terraform daily, I won't agree with you. So many times I had to manually alter state files, and often contribute to basic providers, and wait months to get PRs merged in. It used to be that CloudFormation lagged behind Terraform for AWS, but not anymore. Now, it's quite the opposite and you see people using CloudFormation resource within Terraform as they are stuck into using plain Terraform and yet don't want to manage resource manually.
You won’t find argument from me that HashiCorp have let maintenance slide, but there is still no conceivable way that CFN is more reliable in the general case.
Hmmm, just searched GitHub for aws_cloudformation_stack and found 375 public repositories. I'm sure those references are orders of magnitude more in the private repos. In other words, you still need CloudFormation with Terraform on AWS... so, why use the patchwork then?
The stregth of TF is the community and quality of docs. I use providers to manage an infra that is a mix of AWS + Heroku + external services like Rollbar and that just comes easy with TF. The point of it being paid (although I don't use any of their paid services), is that it is their core business and they are forced to continuously improve on it. TF 5 years ago sucked, but TF in 2021 is super smooth.
The only problem is Cloudformation is strictly AWS resources, so unless you're absolutely 100% sure you'll never need anything outside of AWS ecosystem, I always recommend using TF.
If you're invested, you can use extensions [0] which seem like they're more well-defined. It sounds like this is what APN partners like Datadog use to implement 3rd party resources.
This is true, of course, but you can have a Terraform CloudFormation resource, too, and pass data from other resources back and forth. Since when we don't like using the best tool for the job?!
For companies the likelihood of not needing stuff outside of AWS is relatively low. Unless you're okay with managing other applications by hand, which is probably the wrong attitude.
Honestly, I really like CFN, but raw templates are a bit of a disaster... Most of the rest of my infrastructure is powershell, so I acutally write the templates inline in Powershell something like this:
New-AWSTemplate -Resources @{
$OAI = @{
Type = "AWS::CloudFront::CloudFrontOriginAccessIdentity"
Properties = @{
CloudFrontOriginAccessIdentityConfig = @{
Comment = "Access to the bucket"
}
}
}
....
}
If I were doing something without that context I'd likely use Javascript, and just stitch together template structures
I've tried out CDK but honestly it seems very opaque and hard to understand, I want to give it another shot but it just doesn't seem to be a direct translation from CFN templates...
Having written more CFN than I care to remember CDK is a breath of fresh air. It has a multi-level API that deals with the relationships between resources in a much cleaner way than plain CFN. However, at the lowest level you can still write basically CFN in code. But CDK has so much more. It includes multi-language support, unit testing, compile time checking for errors, etc.
Reading CDK's introduction page, it seems that, similar to CloudFormation, it uses pre-made components, just at a higher level. Those components are then translated into a CloudFormation template[0]. Is that correct?
Might it be more accurate to state that CDK is a higher level version of CloudFormation, that abstracts away unimportant details?
As someone who's used CDK for a few months and never handcoded CF, that sounds completely correct. If you're comfortable with Python, here's a simple but non-trivial architecture you can check out: https://github.com/linz/geospatial-data-lake/blob/master/app....
I couldn't agree more. Cloudformation is a nightmare in my opinion and never should have seen public release. CDK is what Cloudformation always should have been.
CDK is the first time I experienced the benefits of TypeScript. If nothing else it’s a great playground to learn that stack. Testing stacks in the CDK is pretty rad.
This is not correct. CDK is not "managed IaC" - the management is done by CloudFormation. CDK is a generator for the JSON or YAML of CloudFormation, which is fully responsible for the execution and management of provisioning.
To summarize their best practices:
* don't put creds in templates
* reference other parameters outside of the template, e.g. in Systems Manager Parameter Store
* make your code readable
* add comments
* check your code
Except for the second, all of those are fine practices for maintaining any code.
The one best practice that's actually relevant to CF is to store parameters in SSM. Yet, they mention that drift is a problem with CloudFormation. So... why do you want to store parameters to your stack in a service that allows them to change independently of the stack?
Maybe there's an argument for doing this, but they don't explain it. It's just a "best practice."
They don't mention a major use case for CF, namely CI/CD. The article finishes up with a pitch for their template designer.