
Terraform 0.7 released - Kaedon
https://www.hashicorp.com/blog/terraform-0-7.html
======
AYBABTME
Shameless plug: we use terraform a lot within DigitalOcean and I made a
resource provider for our recent release of block storage. It's part of this
release[1]! (I'm happy, sorry!)

By the way, contributing to the project was very straightforward and the
participants involved are super nice. I recommend you send them a patch if you
have an itch!

[1]:
[https://www.terraform.io/docs/providers/do/r/volume.html](https://www.terraform.io/docs/providers/do/r/volume.html)

------
the_duke
For others like me, who don't know Terraform:

"Terraform is a tool for building, changing, and versioning infrastructure
safely and efficiently. Terraform can manage existing and popular service
providers as well as custom in-house solutions."

[https://www.terraform.io/](https://www.terraform.io/)

------
akurilin
What are people's thoughts on Cloudformation vs Terraform for a project that
only ever expects to use AWS?

~~~
skywhopper
Terraform often beats CloudFormation to the punch for support of AWS services.
Terraform is also far more friendly in terms of being able to preview what's
going to happen and roll things back. If Cloudformation fails, it's really
hard to fix, and too often you need to contact AWS to unstick a CF change.
Basically you have no control over the process and you just have a poor web UI
or a poorer API to interact with. When Terraform fails, the newest version
gives you tools to edit and update your state and import existing resources,
and even when things get ugly you are still in total control. In short, I have
no reservations about endorsing Terraform over Cloudformation.

~~~
toomuchtodo
I just wanted to offer the other side. I do devops, we use Terraform, and
we've had times when Terraform took action that was not present in the plan
generated before applying, in production environments. We've also had times
when it'll fail mid-apply, be stuck in an inconsistent state (IAM profiles and
autoscaling group configurations, specifically), and require manual recovery
through modifying or removing resources via the AWS console (although this
might be better in the latest version).

We'll continue to use Terraform at my day job, but I don't recommend it to
colleagues nor clients if they use only AWS resources. Simplicity is the
ultimate sophistication, and adding yet another abstraction does not
contribute towards that philosophy.

~~~
mitchellh
We consider these kinds of issues the highest priority and always work to
resolve this ASAP. The resolutions very often include not only the fix but
protections to ensure that unintended actions don't bypass anything in the
future.

It's really unfortunate you had these experiences and I can't back up the
statement when I say that this isn't normal and every release certainly does
improve stability in a big way. Terraform is a very, very heavily tested (unit
+ acceptance) project and we fight very hard against regressions with multiple
tests per bug in many cases at different levels.

I'd encourage you to continue giving future versions a shot and if you
experience anything like this again, let us know and we'll react to that
quickly relative to other types of issues.

~~~
toomuchtodo
Thanks for the reply Mitchell! Definitely don't want my criticism to come off
as unconstructive. As I mentioned, we'll continue to use Terraform; I'm
optimistic it'll improve as Hashicorp iterates. I'm just naturally
conservative coming from an ops background.

I try to pick boring technologies that are battle tested through years of tech
sector use; there's no glory for being on the bleeding edge, with only pain
when things go south. So goes the ops struggle.

------
doh
I'm the most excited about the _List and Map Types_. It was a huge pain to
concatenate strings in arrays to move them and then parse them again.

~~~
Rapzid
Have you been able to find the docs on how to use them?! I turned up a few
list methods in the interpolation docs, but didn't see how to actually compose
a list. I'm still having to send module outputs as a space separated string
then join(split( it into my IAM policies :|

~~~
doh
Pretty simple.

# you don't need to state the list in here variable "test" { type = "list"
default = ["test1", "test2"] }

module "test_module" { test = "${var.test}" }

# in module test_module variable "test" { type = "list" }

# and now you can use it as ${var.test}

Hopefully it helps

------
_asummers
Does anyone that has used Terraform have any major complaints about it, or
conversely any specific praises for something it does well compared to other
tools in that space?

~~~
scrollaway
I haven't been using it very long but I'll go. My main complaints:

\- Very steep learning curve. It uses its own configuration language (HCL)
which means you have new syntax to learn on top of its overwhelming API.

\- It's stateful, so you have a state file that you have to store and keep
synchronized in your team. There's capabilities for storing the state remotely
but it's very much not ideal. You seemingly can't generate that state file
from scratch based on your current provider's state.

\- The templates are atrociously ugly and the string interpolation is limited,
has extremely rough edges.

\- It's very much an 0.x product. I keep hitting bugs. EC2 machines don't
destroy properly if they have mounted drives. S3 buckets can't be force-
destroyed if they have versioning enabled. Some normalization issues here and
there which cause constant changes to show up.

\- The custom format means the files aren't easily parsed and/or generated
unless you're using hashicorp's own hcl go library. This sucks, to say the
least. TOML with some conventions and jinja2 templating would have done the
job just fine and would have been a ton more readable imho.

It's still good software and design. I would definitely pick it up for new
project, but I wouldn't bother migrating existing ones just yet unless you
know you'll need it. I use it alongside Ansible and the two pair quite nicely.

~~~
atombender
The state management system always struck me as very flawed design, and it's
the reason I've not yet used Terraform.

After all, it means that you _have_ to start from scratch. If you _any_
existing infrastructure — instances, load balancers, DNS, whatever — then
anything you declare in Terraform will conflict, even if it's declared exactly
identically.

A workaround is to manually reverse-engineer your current state into a state
file, or use a third-party tool like Terraforming [1], but it still breaks the
second anyone (either accidentally or intentionally for whatever reasons)
bypasses Terraform and touches the world directly, so Terraform simply isn't
resilient by design. Surely one of the primary uses of a tool like this must
be that if something _is_ modified behind your back, you are secure in the
knowledge that you can whip out Terraform and it will force the world back
into the right shape in a few seconds?

That being said, 0.7 has an import command now, so they recognize the problem.
It's extremely limited and is not a complete solution yet, however.

I never understood the purpose of the state file. A tool like Terraform, from
my perspective, should be stateless. It ought to compare its target to the
world, and then attempt to converge the world to match the target. You could
have local state as an _option_ , in particular to _detect_ interference, but
I don't see why it's needed at all times.

My current solution is to use Salt (not Salt Cloud, just the Salt "boto"
states, using local masterless mode), but it's pretty terrible. I would really
want a tool like Terraform, but without the flawed state management.

[1]
[https://github.com/dtan4/terraforming](https://github.com/dtan4/terraforming)

~~~
mitchellh
The state management is a primary focus for us right now, and I think that's
evident from this release: state management CLI tools as well as the beginning
of import (which is primarily dealing with state in this iteration).

> ... it still breaks the second anyone ... bypasses Terraform and touches the
> world directly, so Terraform simply isn't resilient by design.

This shouldn't be true. If you create a new resource that was never under
management by Terraform, then yes, Terraform will ignore it. This is by design
so that you can use Terraform to manage infrastructure and use other processes
to manage other infrastructure that perhaps isn't quite migrated yet OR
doesn't fit Terraform's model for whatever reason.

However, if you change a resource under management or even remove it,
Terraform will notice this. It is very resilient to detecting drift.

> I never understood the purpose of the state file.

The purpose is to map your resources to what exists in the world. There must
be some way to make a mapping there to be able to do diffing, drift detection,
etc.

Some cloud platforms provide interesting ways to work around this (AWS tags
are often abused for this) and we actually experimented with that approach
early on. Unfortunately, not everything supports tags and there was no way for
us to separate what was Terraform managed and what wasn't.

One of the features we actually have planned for Terraform in the future is
the ability for it to tell you what IS NOT under Terraform management (by
comparing local state to global state). For companies that have everything
under management, this will be a great check to find any rogue resources. For
companies working on adopting Terraform, this will help find things that still
need to be migrated/imported.

So, hopefully our state improvements in 0.7 and in the short term will help
sway you. They're definitely a top concern. But I also hope I explained my way
through some of the design here!

Thanks for the feedback and we hope to see you in the community soon!

~~~
jacques_chester
I think that opening changes in state via a CLI is a fast path to making
things worse.[0]

I greatly prefer the BOSH model: "give me a manifest and I will make the world
look like it".

Documents are easily versionable. Commands aren't.

Documents narrow scope and can be more easily made idempotent as an automatic
guarantee. Commands require us to remember to check the state ourselves,
manually, before doing something.

That said, Terraform has a significantly different low-level model. BOSH looks
at the size of the problem and imposes non-negotiable primitives to scope that
complexity. Terraform is more ambitious. I suspect Terraform will be easier to
adopt incrementally, but harder to manage on a per-unit-of-state basis.

[0] Loosely related: [http://chester.id.au/2012/06/27/a-not-sobrief-aside-on-
reign...](http://chester.id.au/2012/06/27/a-not-sobrief-aside-on-reigning-in-
chaos/)

~~~
mitchellh
> "give me a manifest and I will make the world look like it"

This is how Terraform works if you want a one-time setup. You can just throw
the state away after that if you want. But Terraform is supposed to be used
(and is very very often used) to do ongoing minimal changes to infrastructure,
often dozens of times per day.

In this scenario, Terraform's configurations are completely declarative: it is
a state of the world you want to reach, it isn't what to do next.

However, we require the state file in order to find the resources we own and
then refresh the state of the world so we can do a diff.

~~~
scrollaway
This is a bug we just hit 5 minutes ago on 0.6: One of our CD builds failed
because of a configuration issue - a variable wasn't properly set, leading
terraform to continuously ask jenkins' stdin for a password.

We fixed the bug. But somehow, that previous build wiped the state file. When
we pushed the fix, terraform _freaked out_ and started deleting all our
instances, including S3 redirects that weren't managed by it which we
explicitly told it to ignore.

That sucked.

~~~
irq
Always, always view (and save to disk) tf's plan. And if it looks good, apply
the saved plan output.

~~~
scrollaway
Did view it... locally. It kind of defeats the point if it's not reliable
enough to trust it'll behave the same from a different machine.

No harm done, we're still in beta to catch bugs exactly like those, but our
initial enthusiasm with terraform has been waning very quickly. Like I said, I
love the design, but it's just not there yet.

------
kdeldycke
This release is the first one with built-in support for Scaleway as a
provider:
[https://terraform.io/docs/providers/scaleway/](https://terraform.io/docs/providers/scaleway/)

------
akramhussein
Can't praise Terraform enough. Just so well designed and a prevents you from
ripping your hair out. Great team as well.

------
micah_chatt
Terraform seems like a great project, and the new import feature could nearly
convince me to switch. My biggest barrier to using it is the lack of cross-
account role support with AWS[0].

[0] [https://github.com/aws/aws-sdk-go/issues/472](https://github.com/aws/aws-
sdk-go/issues/472)

~~~
dfinninger
That was a pretty big issue for us as well. We've gone through some
acquisitions and we left with a bundle of different accounts.

Wound up taking a day and writing a ruby script that checked out an STS token,
and exec'ed a new bash shell with the correct environment variables. the $PS1
has the account name in there so it's not too hard to know what shell points
where. Also helpful that we set up a "control account" that handles
consolidated billing and user login.

We've had some pretty good success so far, however I can definitely see how
that feature would be good baked-in somehow.

~~~
JimmyL
We use [https://github.com/atward/aws-profile/blob/master/aws-
profil...](https://github.com/atward/aws-profile/blob/master/aws-profile) to
do pretty much the same thing, although without the $PS1 hacking. Our rule is
that a user's default profile should always be the "control account" (on which
they have permission to do nothing except STS and AssumeRole), and that they
need to use this wrapper and explicitly specify a profile for all "real" API
commands. This also works nicely with MFA.

Having this built into TF would be nice, but there are enough tools out there
that don't support AWS role-jumping that I suspect we'll end up using that
wrapper for a long time.

------
jlgaddis
Terraform looks really nice although it seems it would be of limited use to me
as most of my servers are physical. I do have a some VMs running on VMware,
however. Anyone know what versions of vSphere are supported (I see it's been
tested with 5.5 and 6.0, but I think we've still got a few older ones out
there)?

------
teajunky
Do I need AWS to use this tool? The company I work for operates its own data
center.

~~~
ominous_prime
Terraform isn't specific to AWS. You can see a list of the builtin providers
here that may be useful:

[https://www.terraform.io/docs/providers/index.html](https://www.terraform.io/docs/providers/index.html)

------
GrandTheftR
This is a great release, have been using Terraform for the last year, a life-
saver for us.

Kudos to the TF team/community and Mitchell

