
Ask HN: How do you work with AWS? - wareotie
I&#x27;m using AWS Toolkit for Eclipse (https:&#x2F;&#x2F;aws.amazon.com&#x2F;eclipse).<p>How do you work when you need to develop new things? What is your workflow?
======
carusooneliner
When I'm trying out a new AWS service I start using it through the web
console. Once I gain some familiarity with the service and settle into a
pattern of manual operations, I translate the manual operations into CLI
commands. The most commonly run CLI commands I throw into a shell script.

For instance, I started using AWS Lambda by manually uploading the lambda code
.zip file through the web console. Later on I wrote a shell script that
invokes an AWS CLI command to do the same.

TIP: when using AWS CLI, set the parameter '\--output json' to get pretty,
readable output. It's much better than the default output text format.

~~~
nickthemagicman
What about using Serverless or a framwork for Lambda stuff?

~~~
xenomachina
(I'm not the one you were replying to, FWIW)

> What about using Serverless or a framwork for Lambda stuff?

If you're using other parts of AWS than Lambda, is there a benefit to using
serverless? If I still need to know aws cli for the other services I use,
what's the point in learning a specialized tool that only works for one
service?

~~~
jklein11
Its less overhead. You don't have to worry about keeping the EC2 instance or
web server running/patched/updated/etc. The juice may not worth the squeeze
with the extra complexity of using AWS Lambda.

~~~
xenomachina
Are you comparing Lambda to non-Lambda, or are you saying that in addition to
managing a Lambda function, serverless can also "[keep an] EC2 instance or web
server running/patched/updated/etc"?

------
mbu
I've experimented with a few different things and there really are lots of
valid ways to work depending on context.

Terraform: at some point you will probably end up here. It gives you so much
control, it's fast and (mostly) easy to work with. It gives you a good idea of
what it is doing and why.

Cloudformation: mostly I've used this embedded in some other tool. For example
we have a fairly small Elastic Beanstalk app which uses SQS and SNS - nice to
be able to extend the basic infrastructure easily inline with your app
definition. Similar situation with serverless.

Awless: this is a really nice (scriptable!) CLI alternative. Nothing wrong
with the normal AWS CLI but this does some things simpler/nicer. If what you
are doing is simple enough you can script infrastructure with these tools.

Console: I really use this alongside the other things as I'm building
automation or to explore some new service I've not used before.

Ansible: for configuration management (post terraform apply) we are using
ansible. So far we set up an admin box inside the infrastructure and run
ansible from there. We've experimented with having ansible build the inventory
but currently looking at a bash/jq/awless combo to build it dynamically.

Edit: not ever used ansible to modify AWS infrastructure. We keep
infrastructure separate with a configuration management separate from setting
up the infrastructure.

------
mindcrime
At work, the workflow is largely this: developers hand-create resources in the
DEV environment, get things working, work out kinks, then the DevOps team
comes in and translates everything into their Terraform automation setup, and
then deploys to other environments use Terraform.

Personally, when working in DEV, I do most of my work through the AWS Console,
combined with some use of the AWS Java API. I rarely if ever use the AWS
command line tools.

~~~
bni
Wow. Why not automate it from the beginning?

~~~
mindcrime
We may get there eventually, but that's just the way things were done here
when I started. Unfortunately we have one of those environments where "devops"
is basically a separate department, and are treated more like sysadmins than a
core part of the development team. So the culture leads to a mindset of
"DevOps does the automation stuff".

~~~
echlebek
I've been there. It's a frustrating situation, and difficult to change.

------
devonbleak
"Sandbox" accounts where devs have admin access to go figure out how services
work and fit together.

Once they're ready to go into actual dev environment it's Terraform.

We also have a service catalog for standing up ec2 instances that are fully
managed like in the datacenter and cloudfoundry.

Personally I tend to use a mix of terraform, boto3 and CLI for the vast
majority of things, and log into the console when I need to go look at things
occasionally.

------
albertlie
When developing new things, I always used Terraform and Jenkins.

The reason is because:

1\. Terraform You can abstract the deployment logic and requirements into
single unit because developers ideally should spend more time building
products rather than dealing with the deployment and cloud stuff. Furthermore,
this will be one source of truth for any deployment configuration and
documentation, that will be super worthed in onboading as well especially when
you're working with big engineering team (> 50 engineers)

2\. Jenkins We actually can run terraform from our local computer, but with
Jenkins we can standardize the way people update and create the resource in
consistent way. Also, by using Jenkins we can track all deployment that
happened

My future plan is for combining this workflow with management configuration
tools like Ansible since Terraform is not only for creating cloud resource,
not for manage configuration. Currently, I'm ussing bash command which
obviously won't be easily scalable and understandable by engineers.

------
laurentl
Console to figure out a new service (it’s a quick way to see all the different
configuration options). CLI for some actions not supported by the console.

Then once we’ve figured out how we want to use the service, we script the
deployment with CloudFormation and put it in an Ansible playbook. This allows
us to rebuild everything from scratch, and that’s actually what we do for each
deployment. Also makes it super easy to set up a new environment (e.g. to
onboard a new team member).

We briefly considered Terraform but the learning curve seemed steep - should
we give it another shot?

------
aloukissas
I use the serverless framework CLI
([http://serverless.com/](http://serverless.com/)) 80% of the time. Does all
my config + deployment for me.

------
autotune
Every client has different requirements so it varies. CloudFormation is common
but strongly prefer Terraform when possible for creating infrastructure, and
use Ansible or chef-solo for config management, pre-baked AMI method and
deploying to ASG Launch Config for deployment. Use Python and Lambda for
scripting out things like backups and common security tasks or anything else
that may need to be scripted out.

~~~
guitarbill
I generally love Hashicorp's tools, but never really got using Terraform over
CloudFormation.

I found the cloud-agnostic claim to be a bit of an exaggeration, plus most
clients are happy to stick to one provider. Sure, CloudFormation is a bit
slow, and the JSON is awful, but with YAML it's okay. And I don't need yet
another tool. What am I missing?

~~~
dmlittle
I'm not too familiar with using straight up CloudFormation JSON definitions.
Are you able to create reusable modules in CloudFormation?

In Terraform you're able to create modules that you can use to setup uniform
infrastructure. For example, you could have an RDS module that aside from
setting up the database it also sets up a replica, a dedicated KMS encryption
key, IAM policies, security groups, parameter groups, etc. So instead of
setting all of this up for every database you have, you can just reuse a
custom module that does all of this for you. These modules can also be
versioned such that you can upgrade and change them as needed without breaking
all your current infrastructure.

~~~
guitarbill
Arguably you can create a stack that does this and exports the DB identifier
to be used in other stacks. It doesn't sound as convenient though, and child
stacks have some annoying limitations.

------
bradknowles
Well, cfndsl allows us to write our cloudformation templates in YAML instead
of JSON. Those can also get easily checked into git.

We are also trying out some stuff with serverless.

But we are an AWS-only shop, so terraform and other cross-cloud tools are not
of interest. And we are also not interested in anything that can’t take full
advantage of everything AWS offers.

And we are particularly uninterested in anything based on kubernetes.

~~~
guitarbill
CloudFormation supports YAML templates out of the box:
[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/gettingstarted.templatebasics.html)

It's way nicer than JSON, especially w.r.t. functions. It's also compatible,
so you can load JSON templates into e.g. a Python script and serialize them to
YAML. I did this to all our templates pretty much as soon as it came out.

I have used cfndsl and loath it. If I need more complex variables than
CloudFormation supports, I rather use a simple templating engine like Jinja2
or ERB than a Ruby-based DSL.

------
thepumpkin1979
I use the console to manage S3 buckets and credentials in IAM. EC2 instances
are provisioned via Rancher.

------
the_new_guy_29
Straight into automation using tools like Terraform or working directly with
K8S cluster

------
crazypyro
Gradle, Terraform and Jenkins.

