Hacker News new | past | comments | ask | show | jobs | submit login
Pulumi – Modern Infrastructure as Code (pulumi.com)
380 points by evo_9 50 days ago | hide | past | web | favorite | 146 comments



I have been wanting to try Pulumi out for a while. As an avid terraform user, I welcome the ability to use a proper language instead of the declarative HCL. For those who have migrated from terraform -> Pulumi, what are the biggest cons you've experienced?


I witness the same mentality in sbt users, Turing computable > declarative. I don't get it though.

In both the build system and the deploy system, I want to know the config terminates, and I want it to be easy to understand. The application's specialization is Turing computability. I prefer that footgun to stay isolated there. But maybe there is a use case I don't get.

Imagine accidentally provisioning infinite VMs...


[disclaimer: CTO at Pulumi so clearly biased :-)]

One of the things I really believe is that you can have the best of both worlds here. Pulumi uses imperative programming languages, but is still "declarative". The imperative programs are executed to build up the desired state, which can then be reliably diff'd and previewed, and can be used to enforce manual or automatic checks for correctness. So you get the expressiveness of imperative programs (loops, conditionals, components, packages, versioning, IDE tooling, testing, error checking, etc.), but still the safeguards and reliability of declarative infrastructure-as-code (preview, gated deployments, policy enforcement, etc.).

I also tend to view the perceived benefits of JSON/YAML/HCL "simplicity" as somewhat comparing apples to oranges on a complexity specturm. If you are only managing a dozen resources, it may be that JSON/YAML/HCL are fundamentally simpler. But when you've copy/pasted tens of thousands of lines of YAML around all over your codebase to manage hundreds or thousands of resources, the value of abstraction, reuse, well defined interfaces, and tooling to manage that complexity feels to me essential to the scale of the problem. And that degree of complexity is no longer just something large organizations are dealing with. Modern cloud technologies (serverless, containers, Kubernetes, etc.) are leading to significant increases in the number of cloud resources being managed, and the pace at which those resource are deployed and updated.

Assembly is a "simpler" way to think about programming, but didn't scale as complexity of application software increases. I believe the same is true about JSON/YAML/HCL and cloud infrastructure.


> But when you've copy/pasted tens of thousands of lines of YAML around all over your codebase to manage hundreds or thousands of resources, the value of abstraction, reuse, well defined interfaces, and tooling to manage that complexity feels to me essential to the scale of the problem.

You're lumping in HCL (and languages like Dhall by extension) with static serialization formats and criticizing them for a characteristic only found in the latter.

HCL is programmable and has a fair model for code reusability through modules, state outputs, for-expressions and other kinds of expressions.

Add in a proper language with types like Dhall and you have a configuration language where you can apply all the transformations you could want with a much higher safety and robustness floor than a turing-complete language that allows you to make all sorts of messes.

It's specially dangerous to have a turing-complete language for configuration once you factor in that the reflex of an inexperienced developer who is more likely to make these messes is to use a tool they're already familiar with even when the tool is actively harmful to their goals, as Pulumi facilitates.


[Disclaimer: also member of the Pulumi team.]

We've worked with a lot of end users to migrate from Terraform, and we honestly do see a lot of copy-and-paste. I agree that it's not as rampant as with YAML/JSON, however, in practice we find a lot of folks struggle to share and reuse their Terraform configs for a variety of reasons.

Even though HCL2 introduced some basic "programming" constructs, it's a far cry from the expressiveness of a language like Python. We frequently see not only better reuse but significant reduction in lines of code when migrating. Being able to create a function or class to capture a frequent pattern, easily loop over some data structure (e.g., for every AZ in this region, create a subnet), or even conditionals for specialization (e.g., maybe your production environment is slightly different than development, us-east-1 is different, etc). And linters, test tools, IDEs, etc just work.

For comparison, this Amazon VPC example may be worth checking out:

- Terraform: https://github.com/terraform-aws-modules/terraform-aws-vpc/b...

- Pulumi (Python): https://github.com/joeduffy/pulumi-architectures/blob/master...

- CloudFormation: https://github.com/aws-quickstart/quickstart-aws-vpc/blob/ma...

It's common to see a 10x reduction in LOCs going from CloudFormation to Terraform and a 10x reduction further going from Terraform to Pulumi.

A key importance in how Pulumi works is that everything centers around the declarative goal state. You are shown previews of this (graphically in the CLI, you can serialize that as a plan, you always have full diffs of what the tool is doing and has done. This helps to avoid some of the "danger" of having a turing-complete language. Plus, I prefer having a familiar language with familiar control constructs, rather than learning a proprietary language that the industry generally isn't supporting or aware of (schools teach Python -- they don't teach HCL).

In any case, we appreciate the feedback and discussion -- all great and valid points to be thinking about -- HTH.


> It's common to see a 10x reduction in LOCs going from CloudFormation to Terraform and a 10x reduction further going from Terraform to Pulumi.

I don't see this as such a terrible problem. The configurations may have more LOC's but there are not as many surprises. The dependency of declarable configuration makes it rock solid and favorable among operations teams who need to make these kinds of changes all the time.

> A key importance in how Pulumi works is that everything centers around the declarative goal state. You are shown previews of this (graphically in the CLI, you can serialize that as a plan, you always have full diffs of what the tool is doing and has done. This helps to avoid some of the "danger" of having a turing-complete language. Plus, I prefer having a familiar language with familiar control constructs, rather than learning a proprietary language that the industry generally isn't supporting or aware of (schools teach Python -- they don't teach HCL).

I understand the reason to want this. Having worked closely with developers, lack of familiarity with HCL makes it much less accessible. However, from an operations perspective, I am GLAD that HCL is a very limited language. No imports of libraries all over the place (in your infrastructure configurations, no less!).


> I don't see this as such a terrible problem. The configurations may have more LOC's but there are not as many surprises. The dependency of declarable configuration makes it rock solid and favorable among operations teams who need to make these kinds of changes all the time.

The issue is that your static configs often have lots of boilerplate sections that have to be kept in sync. Further, you can use an imperative language like Python, JS, etc and still write in a completely declarative fashion (or you can use a functional language which tend to be declarative out of the box). Conversely, you can model an AST in YAML (which is what CloudFormation is trending toward) and get the worst of all worlds. Bottom line: don't conflate "reusability" with "imperative" or "static" with "declarative".


> The issue is that your static configs often have lots of boilerplate sections that have to be kept in sync.

Yes, I agree with this. However, its predictable. As an operations person, I value predictability and am willing to pay the price of keeping static configs in sync.

> Further, you can use an imperative language like Python, JS, etc and still write in a completely declarative fashion (or you can use a functional language which tend to be declarative out of the box). Conversely, you can model an AST in YAML (which is what CloudFormation is trending toward) and get the worst of all worlds. Bottom line: don't conflate "reusability" with "imperative" or "static" with "declarative".

Hold on, I'm not conflating anything. Saying that "you can write terrible things in any language" isn't anything new. We choose to use languages that provide certain guarantees that we need for the domain that we're working in. For infrastructure, declarative languages are a lot more suitable for the properties they provide (i.e. no surprises, limited functionality etc.). Its "possible" to use static types in Python, how many do that?


> Yes, I agree with this. However, its predictable. As an operations person, I value predictability and am willing to pay the price of keeping static configs in sync.

I think there's wisdom in this at small scales, but as the volume and complexity of your boilerplate grows, I think you lose any advantages. I also think this threshold is quite low (as an ops person and a dev person) since it's not much harder to look at/read the YAML generated by a script vs that which is hand-rolled and committed to git.

> Hold on, I'm not conflating anything.

Are you sure? Because you just said "I am willing to pay the price of keeping static configs sync" and then "For infrastructure, declarative languages are a lot more suitable for the properties they provide" and then you started to talk about "static types" in Python, which is different than "static" in the YAML sense (YAML isn't statically typed, but it is static in that it isn't evaluated or executed).

I'm not trying to be a jerk, it just sounds like a lot of concepts are being confused. I also wasn't making the argument "you can write terrible things in any language" (not sure if you were attributing that argument to me or if that was a point you were trying to make).

Consider this Python: https://github.com/weberc2/nimbus/blob/master/examples/src/n...

It's fully declarative, but it does evaluate, so it's not static in the YAML sense. It outputs a JSON CloudFormation template (but it could easily output in YAML) which you could inspect visually before passing onto CloudFormation.

It's also statically typed although that's not evident from this file since all types are inferred in this file (however there are annotations in the imported libraries), and while the static typing is a very useful property, it's not what I've been talking about in this thread.

In my opinion, this is no less readable than the equivalent YAML; however, it's capable of doing much more (albeit if your infrastructure is just one S3 bucket, then this is overkill--to really understand the power of dynamic configuration, you would want a more complex example).


I can’t trust my teammates to write code that doesn’t use raw eval()’s all over the place.

Getting them, nevermind relying on them to write Python/JS in the correct way is straight up out of the question.

At least I know in Terraform/HCL they can’t map a config change over the 1000 new instances they spun up because they happened to write their for loop wrong.


> I can’t trust my teammates to write code that doesn’t use raw eval()’s all over the place. Getting them, nevermind relying on them to write Python/JS in the correct way is straight up out of the question.

Then use Starlark (https://go.starlark.net) or Dhall or similar.

> At least I know in Terraform/HCL they can’t map a config change over the 1000 new instances they spun up because they happened to write their for loop wrong.

To be clear, the proposal is to use a programming language to generate your HCL-equivalent configs, not to imperatively modify infrastructure. Consequently, you can inspect the generated "HCL" (or whatever the output is) and make sure it looks like the code they would write manually. Further, you can even write automated tests.


So, things need to be comprehensible by the humans that work with them. A 10x reduction in LoC / 10x increase in expressibility may or may not be a good thing, but if it captures intent better and with less ceremony and cruft, then it most decidedly is a FANTASTIC thing. Whereas a 10x LoC improvement that makes it harder to glean intent would be DISASTROUS.

Then again, code has to be run in order to analyze its output -- that or code has to be data you can analyze (like a Lisp), but that can be very difficult to reason about.

So my preference would be to have libraries for constructing configuration data. Then you can execute a program to generate the configuration, and that you can use without further ado. The output may not be easy for a human to understand, though it should be possible to write code to analyze it.


"For comparison, this Amazon VPC example may be worth checking out"

It might be better to compare how to use the module/stack:

- Terraform: https://github.com/terraform-aws-modules/terraform-aws-vpc

- Pulumi: https://github.com/joeduffy/pulumi-architectures/tree/master...

So as a user, can I configure this Pulumi VPC stack before it's instantiated? Or do I have to use the defaults first and then use the CLI to change things? Do these CLI changes then get placed into code, or just into state? Does that mean I'm now in a situation where the code doesn't match the state?

Personally I find the Terraform configuration much easier to reason about, I see exactly where resources are declared just by scanning the file. (But I've also used Terraform a lot).

Edit: Ah, maybe I have to configure it via this config.py file [1]? I appreciate what Pulumi is trying to accomplish, but that is certainly not a config format I'd like to be using. Maybe you could use HCL or YAML for it? ;)

Edit 2: Another last thought, I think a lot of the mindset in Terraform comes from Go, where the proverb "A little copying is better than a little dependency" is pretty well adopted. Before I started writing Go as my main language I didn't appreciate that mindset, but after 5 years with Go I've found it more and more appropriate [2].

[1] https://github.com/joeduffy/pulumi-architectures/blob/master...

[2] https://go-proverbs.github.io/ -- https://www.youtube.com/watch?v=PAAkCSZUG1c&t=9m28s


You're right, the Pulumi example is a project, not a reusable module. There are a few approaches to making it modular:

1) The project does support config. So if you want to change (e.g.) the number of AZs, you can say

    $ pulumi config set numberOfAvailabilityZones 3
    $ pulumi up
And Pulumi will compare the current infrastructure with the new goal state, show you the diff, and then let you deploy the minimal set of changes to bring the actual state in line with the new goal state. This works very much like Terraform, CloudFormation, Kubernetes, etc.

2) You can make this into a library using standard language techniques like classes, functions, and packages. These can use a combination of configuration as well as parameterization. If you wrote it in Python, you can publish it on PyPI, or JavaScript on NPM, or Go on GitHub -- or something like JFrog Artifactory for any of them. This makes it easy to share it with the community or within your team.

3) We offer some libraries of our own, like this one: https://github.com/pulumi/pulumi-awsx/tree/master/nodejs/aws.... That includes an abstraction that's a lot like the Terraform module you've shown, and cuts down even further on LOC to spin up a properly configured VPC.

I am a big Go fan too, so I very much know what you're saying. (In fact, we implemented Pulumi in Go.) Even with Go, though, you've got funcs, structs, loops, and solid basics. Simply having those goes a long way -- as well as great supporting tools -- and you definitely do not need to go overboard with abstraction to get a ton of benefit right out of the gate.

Again, I'm biased and YMMV :-)


"The project does support config. So if you want to change (e.g.) the number of AZs, you can say..."

Cool, is it possible to do that without having to use the CLI? Are you doing any sort of state locking here? I've seen ops teams get saved from potentially horrible situations by Terraform's dynamodb state locking.

"You can make this into a library using standard language techniques like classes, functions, and packages."

That's pretty nice and it seems like it'll get you the same functionality as a Terraform module. Do you have any plans of releasing something like the Terraform Registry to help with discoverability?

Also, do you have any docs on writing providers? I've had to do that a few times for Terraform and getting up and running with that was pretty easy as a Go developer. I wouldn't really want to do that for every supported language though (no offense C#).

I'm seeing that some of this is using codegen to read the equivalent Terraform provider and generate the Pulumi provider from that schema. Is that the preferred workflow here for providers that already exist in the Terraform ecosystem?


> is it possible to do that without having to use the CLI? Are you doing any sort of state locking here?

Yeah it's just a file if you prefer to edit it. By default, Pulumi uses our hosted service so you don't need to think about state or locking. That said, if you don't want to use that, you can manage state on your own[1]. At this time, you also need to come up with a locking strategy. Most of our end users pick the hosted service -- it's just super easy to get going with.

> Do you have any plans of releasing something like the Terraform Registry to help with discoverability?

I expect us to do that eventually, absolutely. For us it'll be more of an "index" of other package managers since you already have NPM and PyPI, etc. But definitely get that it's helpful to find all of this in one place -- as well as knowing which ones we bless and support.

> Also, do you have any docs on writing providers?

We have boilerplate repos that help you get started:

1) Native providers: https://github.com/pulumi/pulumi-provider-boilerplate

2) Terraform-based providers: https://github.com/pulumi/pulumi-tf-provider-boilerplate

These packages are inherently multi-language and our code-generator library will generate the various JavaScript, Python, Go, C#, etc, client libraries after you've authored the central Go-based provider schema.

> Is that the preferred workflow here for providers that already exist in the Terraform ecosystem?

Yes. We already have a few dozen published (check the https://github.com/pulumi org when in question). In general, we will support any Terraform-backed provider, so if you have one that's missing that you'd like help with, just let us know. We have a Slack[2] where the team hangs out if you want to chat with us or the community.

[1] https://www.pulumi.com/docs/intro/concepts/state/#self-manag...

[2] https://slack.pulumi.com


I would point out that Dhall solves these problems with one simple fundamental construct: the function.

And still keep it terminating.

You can do it but it means doing more cognitive engineering than "just throw python at it".

Another point: you can have a declarative turing complete language. I would really like to see people bring prolog like languages to things like pulumi and terraform.

That would also allow to get convergent concurrent application which means we could get proper collaboration. That would be a strong move ahead for devops.


> We've worked with a lot of end users to migrate from Terraform, and we honestly do see a lot of copy-and-paste. I agree that it's not as rampant as with YAML/JSON, however, in practice we find a lot of folks struggle to share and reuse their Terraform configs for a variety of reasons.

I would risk to say that it’s not the Terraform that makes the people to copy / paste. It’s the people. Call it lack of knowledge, not enough time, laziness, tight schedules...

Once your customers are on their own, new people join - no knowledge of Pulumi, resources get added / moved / evolve, there will be copy / paste in their Pulumi code too.

Not defending Terraform here. Just adding a point to the discussion.


Some of this is truly on terraform. The for construct (and looping in general) was only added in TF 12, released in May 2019. Older codebases didn't have a real way to support looping so there's more copy paste there. TF supports ternary conditionals, but not true if statements, which makes adding more complicated if logic difficult.

The reality is that all programming languages have significant copy paste codebases using them, but there are features which help reduce the amount of it. Terraform is missing some of those features, and many of the features it does have were introduced in tf 12, which is less than a year old.


Yes. But Terraform (hcl) is not a programming language.

It’s interesting that some people bring up sbt as an example of how to use a „programming language” for configuration. The reason why sbt became dominant was the weight of Lightbend (Typesafe). There was no way to get away from it. Frankly, sbt can be awful mashup of copy / paste too. sbt is so much magic, I would not be surprised to discover that majority the folks who use sbt, have no actual clue why stuff works the way it works.

I haven’t tried Pulumi yet, I will try when I get the chance. I am eagerly waiting for an opportunity to use it. Hopefully it will surprise me in a positive way. Surely, it can deliver on what it promises. I have very fond memories of Chef and cookbooks in Ruby, it can be done.

Edit: personally, Chef solo (with right tooling to eliminate the server), was the best experience so far. If Pulumi can improve on that (no agent), I’m looking forward to take it for a test drive.


> I would risk to say that it’s not the Terraform that makes the people to copy / paste. It’s the people. Call it lack of knowledge, not enough time, laziness, tight schedules...

Well, the problem is that a majority of people don't want to / don't have the time to learn HCL, because it's not the most effective use of their time / not worth the "investment" to do so.

Learning HCL is not very rewarding, unless you are an ops person. Learning a general purpose language language like Python, TypeScript or whatever language your company uses is rewarding both for ops and dev people (or devops people if you like that term) and typically can be used for a much wider set of use-cases.

When introducing a new language the pros and cons of doing so should always be carefully considered, however unfortunately for devops tools new languages like HCL,Jsonnet,Starlark,zillions of YAML pseudo-programming DSLs etc. are often introduced very lightly, mentioning a handful of use cases where the new language shines, but ignoring the cons and intrinsic costs (learning curve, new tools, editor integrations, package manager etc. to be built).

Terraform works great for teams where you have a strict separation between ops and dev people. The ops people will spend their time learning HCL, the dev people will learn Python, TypeScript or whatever that is. However if you are trying to truly embrace a "DevOps" model Terraform shows its flaws. Developers will either still heavily rely on ops people to "help them" even for trivial infra changes or they will write sub-par copy pasta HCL code that tends to be verbose.

TF 0.12 may have a bunch of new constructs which make it easier to reduce duplication, but the boilerplate that is required to create an actual reuse module with variables and import it (and overall awkwardness of the module system/syntax compared to any other language) vs the simplicity of creating a reuse function/file in Python/TS is like night and day. Furthermore the subpar editor support for TF makes it actually hard to follow references between modules and safely refactor code, so there is a much lower threshold at which an abstraction appears "magic"/incomprehensible in HCL, compared to typed TS/Python where you can easily follow references.

Source: ~2 years worth of Terraform (incl. 0.12) and ~1 years worth of Pulumi use within multiple companies and teams.


Looking at your Terraform and Python scripts I see two different scripts doing different things with different abstraction levels and different configuration toggles.

It's ironic that you sell as a plus that Python allows you to easily loop over data structures and make resources codnitional, because pretty much all your Terraform resources there are conditional (with a few looping over lists for DRY purposes), while few of the Python resources are.

Many of the lines saved for declaring identical resource types are just because either the Terraform resource is declared with unnecessary values or because the Python one has a default value, which can be provided as well in Terraform.

But yeah, the bulk of the difference is that the scripts are doing different things by declaring different sets of resources.

> Plus, I prefer having a familiar language with familiar control constructs, rather than learning a proprietary language that the industry generally isn't supporting or aware of (schools teach Python -- they don't teach HCL).

Which comes back to my point about inexperienced (or the "10x" ones that cut corners until the table is round and then leave) developers preferring familiarity over using a specialized tool that takes into account common pain points, further fragmenting the space through "worse is better". I am certain I will die employed on cleaning up ORM messes left by developers that didn't want to learn SQL despite having a whole field of mathematics backing it; so if you're successful, odds are I will also end up fixing some day the "declarative output" a Pulumi script produced in a developers computer that is not reproducible anywhere else because it makes a request to his home server and mutates an array of resources somewhere depending on that response, the current time, the system locale and the latest tweet by Donald Trump.


"Many of the lines saved for declaring identical resource types..."

Yeah, it seems a bit silly to say that a benefit is saved lines of code, yet the Terraform example is setup to do quite a lot more than the Pulumi example. The resources are just there and turned off with the "count" configuration. The Pulumi example isn't doing any of the RDS, Redshift, Elasticache, Database ACL, VPN gateway, etc things. This example is a pretty substantial module and I'd guess the LOC would be pretty similar between the two if the functionality were closer.


> It's specially dangerous to have a turing-complete language for configuration once you factor in that the reflex of an inexperienced developer who is more likely to make these messes is to use a tool they're already familiar with even when the tool is actively harmful to their goals, as Pulumi facilitates.

"Turing complete" is a red herring. You can write a program in Dhall that will continue to run long after we're all dead. But this doesn't happen in practice and/or when it does we notice something is wrong fairly quickly and correct the problem. And because these infra-as-code-and-not-configuration solutions generate configuration, if you do have a loop that doesn't terminate or similar, it's not a problem because your program never deploys any changes.

As for making messes, our experienced developers make more of a mess with static configuration because it's fundamentally impossible to manage large static configurations with their inherent repeatable segments that must be kept in sync. The static configuration players try to solve for this by introducing hacky mechanisms for reuse (macros and nested-stacks in CloudFormation, text templates via Helm for Kubernetes, etc), but these fall over very quickly as hacks do.


> "Turing complete" is a red herring. You can write a program in Dhall that will continue to run long after we're all dead.

It's not the avoidance of the halting problem the reason these languages are better for the task. It's the benefit of having limitations that come with being turing incomplete that prevent us from doing a lot of stupid stuff without realizing it and doing "hacky workarounds" without properly understanding the problem we face.

> As for making messes, our experienced developers make more of a mess with static configuration because it's fundamentally impossible to manage large static configurations with their inherent repeatable segments that must be kept in sync.

Or don't do static configuration and just use something like Terraform where you can just reference a resource and pass it around.


> It's the benefit of having limitations that come with being turing incomplete that prevent us from doing a lot of stupid stuff without realizing it and doing "hacky workarounds" without properly understanding the problem we face.

You'll have to articulate your said benefits to be sure, but I would wager that the principle reason to be turing incomplete is to address the halting problem and that the benefits you're thinking about come from other properties of the language (functional purity, immutability, limitations on I/O, type safety where applicable, etc).

Notably, there are lots of hacky workarounds employed in HCL and YAML because people don't understand the problem properly. The problem requires that we can generate arbitrary static configuration from a fixed set of inputs. If your organization is so inept that they keep adding in infinite loops and/or I/O, then by all means, try something like Dhall or Starlark (unfamiliar vs not-type-safe, pick your poison); however, if this is a consistent problem in your organization you probably need to replace your humans because these programs aren't hard to write correctly.

> Or don't do static configuration and just use something like Terraform where you can just reference a resource and pass it around.

Because this only addresses reuse at the resource level. You can do the same thing in CloudFormation; it's not adequate. For example, not everything is a resource. You ultimately need the ability to generate arbitrary static configuration. Terraform probably has lots of other disparate features that collectively address a good portion of the solution space, but programming languages have a unified concept ("functions") that satisfy the whole solution space and programmers are already familiar with them. Terraform's job should be taking static configs and applying them to infrastructure--let a real programming language generate those configs, or at least offer dynamic configuration language that is designed with a proper understanding of the problem (to use your words).


I do not doubt Pulumi is more expressive, but that's also my point. It will be interesting to see how it works out. Sbt won Scala mindshare, there is definitely a strong fanbase for expressivity.

None of my Terraform projects are 10k lines long. I find it's reusable and at almost the right level of abstraction (Typed templates). I tend to go for a minimum expressivity necessary for DRY. So far I have not found Terraform lacking for a single project, but I have found it lacking for expressing higher order infrastructure (infra code intended for multiple projects).

I've never managed a project with thousands of hetrogeneous resources though. I question whether that's really a thing that a single team would do.


I've seen Terraform state files with over 50,000 resources in them. It definitely is something multiple single teams do.


Thanks for the elaborate answer!

I wonder if you considered Dhall (https://dhall-lang.org/) that's declarative but at the same time has functions and other convenience factors.


Not only that, but even with a Turing-non-complete language that lets you do useful things (think of Dhall or whatever it's called), chances are pretty good that you can still take forever to terminate if you really try -- you can provably terminate and you can provably not terminate in a lifetime.

Granted, no one is going to really do that. And there are good reasons to want provably-terminating programs (e.g., in DTrace, eBPF, ..., because probe actions have to not just terminate, but also run very fast). But for infrastructure deployment? I think Turing complete is fine for that.

One idea I've entertained is to use jq as a configuration language and have its output be a JSON text describing a fully-constructed configuration. Yes, jq is Turing complete, but it's so damned convenient!


From the first look of it will still prefer Guile and Guix deploy or Nix with Nixops. All these systems (like pulumi, terraform, ansible and many more) are not really new or innovative, it’s just re-inventing the same wheel with different names and jumping between declarative and imperative syntax.

Guix and Nix both are innovative way to build production, reproducible, secure deployments and platforms without side effects and get rollback and transactions free.


One my of cases was that I just wanted to extend the workflow of my terraform declaration and add a couple of log statements here and there. I didn't want to write Go to write a plugin for that.

With Pulumi - i was able to just sprinkle in a couple of console.log statements and my "extension" was done.


Hand writing JSON/YAML/HCL beyond an initial POC or very, very simple tasks seems like a bad smell.

Using whatever language your project already uses to then dump to these formats is a way more sane pattern IMHO.


WORD


Agreed, I don't really see the upside of getting a "proper" programming language for this.

I just got done with a client of mine migrating a considerable amount of infrastructure code that used Troposphere (Python -> CloudFormation) into Terraform 0.12. The old code was very difficult to reason about, had all sorts of inter-related components that caused infrastructure changes to be scary, and really there seemed to be no benefit of being able to write Python for it.

They now have over 100 services in Terraform 0.12 and they feel confident even making network routing changes in prod during the middle of the day with the new system (something that was unheard of before).

I've found it much easier to write declarative configs in Terraform over the years. Back when I was at Engine Yard we used a project called fog to write infrastructure code in Ruby. It was nice for the time (must have been 2012 or so). But again, I really don't think the problem is that I need a programming language to define my infrastructure. I need a way to declare my infrastructure and manage that state so that I know that what I declare is what is running.

I totally can see teams choosing to write infrastructure code in TypeScript, and Go, and Python. And now you have a mess.


The idea is reuse. YAML (and probably Terraform, though I can't speak to it directly) doesn't give you many facilities for reusing blocks of config, especially if they vary subtly in some parameterized way. CloudFormation gives you some reusability in the way of nested-stacks and macros, but it's seriously heavy-handed.

We do use Troposphere in a handful of cases, and it has its own problems, mostly in that it makes it hard to write declarative Python code with it (which is generally what you want--declarative code but with more expressive power than YAML). I have a prototype of an improvement to Troposphere that I built for my own amusement, and I think I'm on to something:

https://github.com/weberc2/nimbus/blob/master/examples/src/n...

Note that this example is type-safe and declarative while Troposphere is not.

Basically I don't think Troposphere is a good representation of what infrastructure-as-code(-not-yaml) could look like. Not sure about Pulumi as I haven't tried it. But I know the answer isn't YAML, it's not hacking an AST on top of YAML a la CloudFormation, it's not a different static dialect with its own dynamic hacks a la HCL, and it's not generating YAML with text templates a la Helm.


"The idea is reuse."

That's what Terraform modules are for. If I need to bundle up things for, say, multiple environments, I can just write it as a module and configure it through the variables I've exposed.

There's also a registry of community maintained modules if there are some common patterns I don't want to write on my own (https://registry.terraform.io/browse/modules).


I can't speak to Terraform because I haven't used it, but I'm skeptical that modules are sufficient for many of the same reasons that CloudFormation templates are insufficient. I think at some level you need the ability to programmatically generate arbitrary static configuration, and you end up needing something very like a real programming language to do that.

If you need something very like a real programming language, you should just use a real programming language instead of a tool that accidentally reinvents programming language concepts and the corresponding unnecessary learning curve that that implies for developers. I think the reason these tools are surviving in the "marketplace" is because they market is inexperienced and they conflate "generating static configs (to be passed into a tool that can apply the configs) with an imperative language" with "imperative provisioning of infrastructure". The market sees static/declarative solutions like Terraform and CloudFormation as the only alternatives to "imperative provisioning of infrastructure".

I think as our industry gets more experience and tools for generating YAML/static-configs improve, it will be clear that these are the ways forward.


"I can't speak to Terraform because I haven't used it"

Well, then maybe you should try it before you decide it doesn't work! I'll definitely be giving Pulumi a try, but honestly Terraform works really well and I'm happy using it.

It's a nice idea to be able to have a programming language you're familiar with available to do infra work, but deploying and managing infrastructure is a totally different problem domain than writing an application. If the concern is letting developers that don't have much ops experience architect an infrastructure, or you have recent graduates that "don't know HCL", then you have quite a higher learning curve than you actually think you do.

I've never once thought "I really wish I could just use TypeScript for configuring my infrastructure". If that's you, great! Call me when you hit scale and your ops team needs to refactor it.


> Well, then maybe you should try it before you decide it doesn't work!

Based on the assumptions I called out about it, it necessarily can't work as well as a general purpose language. I intend to try it to see if my assumptions are correct or not.

> It's a nice idea to be able to have a programming language you're familiar with available to do infra work, but deploying and managing infrastructure is a totally different problem domain than writing an application.

It's not a hard domain. Developers can write a program that evaluates to YAML--it's not hard. The hard part is applying that YAML to the infrastructure in an efficient way, but programmers don't need to worry about this because Terraform, CloudFormation, Kubectl, etc do it for them.

> If the concern is letting developers that don't have much ops experience architect an infrastructure, or you have recent graduates that "don't know HCL", then you have quite a higher learning curve than you actually think you do.

"Learn Ops + HCL" is a bigger learning curve than "learn ops". Notably, learning two things at the same time is a bigger problem than learning them both individually. But you're right that using a standard programming language isn't a substitute for learning how to architect infrastructure--I never said it did--only that it removes the unnecessary complexity and unfamiliarity imposed by HCL.

> I've never once thought "I really wish I could just use TypeScript for configuring my infrastructure". If that's you, great! Call me when you hit scale and your ops team needs to refactor it.

Ha! If the industry hasn't moved past Terraform in 5 years time, it will only be because HCL has adopted enough of the general purpose programming language featureset as to remain competitive. That's certainly its trajectory.


"It's not a hard domain."

Are you really arguing that deploying and managing infrastructure is not a hard domain? Please. The very existence of tools trying to solve these problems speaks to this difficulty.

I'm going to drop it at this point because it's starting to feel like you're either trolling or have never run a substantial application in production before.


It seems like you’re violently agreeing with me. Like I said, rectifying the infrastructure is the hard part and generating YAML is not. Let the tools do the hard part. They’re good at that. Not so much at the expressiveness and reusability part.


> Ha! If the industry hasn't moved past Terraform in 5 years time, it will only be because HCL has adopted enough of the general purpose programming language featureset as to remain competitive. That's certainly its trajectory.

Quite a statement to make for somebody who has never used Terraform.


I've been keeping tabs on it for a while. Feel free to point out where I'm mistaken.


[flagged]


Yeah, I meant that Terraform is more rigid with respect to reusability than a general purpose language like Python. This isn't a controversial point; the controversy is whether that's a bug or a feature. I think it's a feature, and to the extent that Terraform is becoming increasingly flexible, I would say I'm vindicated in my position.


I think the person above is mostly pointing out about dynamic blocks , since that do allow abstracting blocks of config and also allow subtle differences


want to point that ever since Terraform 0.12, all terraform params and modules etc are transformable in objects, what I mean by that is you can use functions like yamldecode or jsondecode to decode any yaml or json into terraform objects and pass into modules or resources or locals, also "real" programming language is always funny to me when ops are going to be much more comfortable with HCL. JSON/YAML are not in the same class as HCL. YAML maybe can do nesting with & but they can't just create simple config style interface, where the user facing side is a simple yaml file, and the terraform code underneath take it in and pass it to its modules


> want to point that ever since Terraform 0.12, all terraform params and modules etc are transformable in objects, what I mean by that is you can use functions like yamldecode or jsondecode to decode any yaml or json into terraform objects and pass into modules or resources or locals

This sounds like an important development for Terraform, but it also sounds like it's proving my theory that TF is reinventing a standard programming language badly.

> also "real" programming language is always funny to me when ops are going to be much more comfortable with HCL.

Maybe (and I think it's a big maybe), but if you want to empower developers to do their own ops (read "DevOps"), then HCL is a strict loss over a language they're already familiar with.

> JSON/YAML are not in the same class as HCL. YAML maybe can do nesting with & but they can't just create simple config style interface, where the user facing side is a simple yaml file, and the terraform code underneath take it in and pass it to its modules

Right, as previously mentioned, HCL is accidentally reinventing a programming language. The YAML-based infra-as-code solutions also reinvent programming languages but they build them by building out programming language features on top of YAML (instead of extending the language layer itself) or by generating YAML via text templates. If I had to choose between these, I would pick HCL for sure, but thankfully I can just generate static configs with Python or similar.


One use case that TF really frustrates me with is anything conditional: if I want to do blue/green the best I can do is duplicate the entire infrastructure and wrap my terraform invocations with some brittle script that repeatedly runs TF with various targets. The script only allows me to define the target state, but it is quite common that I care how that final state is reached.

Create-if-not-exists is also really poor in TF (probably by design), if you want to reuse your TF configs for different environments in the same account, you either have to ensure everything won't collide name-wise, or split your TF into immutable infrastructure and really-immutable infrastructure.


Doesn't terragrunt allow you to do that?

The looping code while better with terraform 0.12 is still quite nasty though.

https://terragrunt.gruntwork.io/docs/getting-started/quick-s...


The idea of having to use a 3rd party tool to generate config for my 3rd party tool that generates config just seems Wrong.

At this point I'd rather write idiomatic but repetitive Terraform and know that I will be safer on upgrades, as long as I continue to refactor and clean up my modules for the new functionality Hashi is releasing.


I wasn't aware of this, looks like it would solve some of our pain points, thanks!

I think the existence of tools like this is a succinct way to say that static/declarative languages are really hard to execute.


The model is "use a programming language to generate static configs", so you can't provision infinite VMs--your program would OOM or run out of disk because you're trying to generate an infinitely-large config file. If you're seriously concerned about this (and you shouldn't be), you can use something like Starlark (https://go.starlark.net), which is a Python dialect that prohibits unbounded loops, I/O, etc. Note that if you use Python with type hints, you can actually get more safety than YAML.


You can’t step through declarative code in a debugger. That’s a problem for discoverability, which also means more friction for new learners.


You've already lost big if you ever need a debugger to understand code. Especially if your beginners have to use it just to wrap their head around wtf is happening.

And having to run your "config files" just to see what they're doing is the huge downside of using programming languages as config. In other words, it's kind of circular to point out that using code for config puts you in a situation where you might have to use a debugger to know what your config does.


You’ve never told (or been) a new employee to step through the code to figure out what isn’t in the docs?

There’s a middle ground that you see with a number of test frameworks, Gulp, SCons and I suspect with Pulumi, where the outer shell of imperative code is in practice declarative, and the inner bits are properly imperative.

Part of the code says why to do something, and the rest says how to do it. For instance I don’t see a huge conceptual difference between disabling or filtering a test, versus not deploying a service because it is running, current, and healthy.


> You’ve never told (or been) a new employee to step through the code to figure out what isn’t in the docs?

To me that sounds like a failure state.


But isn't the output of this declarative config files?


Now? You're probably right.

If you proxy to replace it might not have to stay that way. But the Lava Flow Antipattern is what happens when momentum fails before the work is complete.


Most sbt users hate it btw.


What is sbt?


Scala build tool. Build files are a mix of scala code and dsl


No built-in remote state backend storage (s3, gcs, etc) locking except their proprietary hosting. Obviously they need to make money, but this makes driving migration from terraform to pulumi a lot more difficult.


I've got a PR opened to implement this but due to S3 CAP limitations, it hasn't been merged. The current state is that it could be merged for non-S3 backends and a DynamoDB type lock could be added for S3 backends. I don't have much interest to keep pushing it forward (its been nearly a year of "ill get back to you next week") although their new VP has been more responsive.

Alternately I was considering just implementing the server API to do state storage & locking, but we have implemented some workarounds that are good enough for now.


I just want to say thanks for your initiative on opening https://github.com/pulumi/pulumi/pull/2697. (I've seen you active on a few other issues and in the Slack channel too)

I think it's really unfortunate (whatever the reason) that the team was this slow to provide meaningful feedback on the PR.


This was true when it was first released, but they now allow you to bring your own backend store.

https://www.pulumi.com/docs/intro/concepts/state/


That page lists this as a feature of their service, implying it isn't available in other backend stores:

Concurrent state locking to prevent corrupting your infrastructure state in a team environment


It's specifically the state-locking feature that's missing from 3rd party backends.


Less community support, due to popularity.

Yes, it's nice to write it in code, but you can achieve similar with raw Terraform and there are more documented examples for that, plus if it goes wrong, you can find others facing the same issues, usually with resolutions/workarounds etc.

I wanted to find Pulumi more useful than I really did.


I've used Terraform quite a bit (though less in the past year), and the things that ended up being the biggest headaches for me were:

1. State quickly gets unwieldy and terraform plans get very slow. I think the official recommendation is to split it up to separate instances using separate state files, but that adds complexity of finding the right seams and sharing any needed data between them.

2. Desperately needs a good way to do singletons. Many modules end up fitting the pattern that you need 1 each of some underlying resources (iam role, iam policy, s3 bucket, kms key, whatever) that could be shared by every instance of the thing you're creating. But you have to decide between keeping the code clear and organized and putting those things in the module next to the resources that rely on them, but then duplicating these things N times unnecessarily (which adds to the problem of bloated, slow state). Or you keep the shared resource outside of the module, which makes the code less clear, bloats the global namespace/tf files and requires passing around even more state variables to the module.

3. Tools for abstractions and structuring the app are lacking in general in other ways. Modules are the only thing you have and they were intended to be standalone for sharing code externally and are clunky when using them only within your own app to organize code. There's no way to import shared constants or even group related variables together into something like structs, so you end up needing to pass around the same handful of individual variables into almost every module and resource you create.

4. Your whole terraform config is only as stable as the worst implemented resources. I think this problem is even worse for smaller, community support-only providers but even within AWS, it's not uncommon to come across bugs in how different services are integrated into terraform. Sometimes useful parameters are missing, or terraform doesn't validate them the same way as AWS itself causing something in your apply to fail halfway through, or things are implemented such that the subsequent tf plan still doesn't show up clean after a successful apply. TF has gotten more stable over time, but there still seems to be an endless long-tail of leaky abstraction type issues like these.

This turned into kind of a long list of gripes, but I'd be very curious to hear from anyone who has used both Pulumi and Terraform in production how Pulumi compares on these painpoints in particular.


> 1.

Start with one per environment. As it grows, you can identify the reusable patterns and move those to their own state files and import them wholesale with terraform_remote_state. It's code, treat it like code.

> 2.

A module for shared resources, a module for resources that use them. You don't bloat anything if you just namespace it and then pass the shared resources' output as an entire object to a parameter in the resources that share them.

> 3.

Same answer. You can create objects for your outputs where you can easily namespace things.

> 4.

This is true. But there's no better way for obvious reasons. It's a high level abstraction, the only alternative is to use low level abstractions, at which point you're just doing shell scripts. Which, by the way, Terraform has pretty clever ways to integrate. It's not pretty, but it's isolated ugliness much like unsafe Rust/C# where you really need to instead of having to go full ugly.

Also, you can always submit pull requests to Terraform providers. Doesn't guarantee the maintainers will be active enough to respond and upstream it, but it will solve the issue cleanly for you and other people at least in the meantime.


Perhaps i'm misunderstanding but this seems to convert a declarative workflow into an imperative one. Doesn't this seem backwards?

Kubernetes manifests are declarative by design and for good reason, the controllers reconcile the state not your script.

Interested to hear Pulumi examples where this is desired and where it fits in with other tools.


[disclaimer: I work at Pulumi]

The Pulumi engine uses a desired state model[1]. You can use imperative language features like loops and conditionals, but all of that gets mapped to a declarative resource graph prior to diffs/updates.

[1] https://www.pulumi.com/docs/intro/concepts/how-pulumi-works/


Thanks for sharing! Interested to try this out


There is no real world site reliability benefit to declarative over imperative. They are just software design paradigms. You still have to design the software to not suck.

People spend loads of engineering effort (years, really) to get that controller to function properly under every weird state case, while the neckbeard with a shell script, hot failover master-master write nodes and an ELB gets five nines without needing a CS degree.


I think the benefit is not needing to grok, debug or re-invent every neckbeard's special shell script. Just declare your desired state and move on to other things.


Agreed.

What concerns me somewhat about Pulumi is we will lose some of "standardization" that comes with the widespread use of Terraform.

Like anything else, it is possible to write Terraform in an incomprehensible way. However, overall I have found that Terraform "constricts" things into being done a certain way. This allows engineers to jump into a new TF codebase and grok what is happening fairly easily.

Now with Pulumi, we lose that constraint, and now have to grok not only the different languages that infra is defined in, but also the wide variety of different "styles" that a language can be written in. An engineer joining from another company or team now has to deal with a new language and a new code style/organization in order to wrap their head around the infra. Not a desirable property when it comes to managing the infrastructure for a company.

While it is appealing that a developer can use a language they are already familiar with to manage infra, that only holds true if the language being used for Pulumi is one they are already familiar with.

I could of course be totally wrong. Regardless, I am excited to see a new contender in the space.


Some of the Terraform codebases I have worked on have been wildly over-complicated and take a good hour to grok. And I literally spend 10 times more of my day writing a new "standardized" Terraform module than I do writing a shell script that does the same thing.

And Terraform doesn't even support declarative configuration management!!! The syntax is declarative, but it literally can not change the infrastructure to be the way you describe, if it has changed outside your state file (which can happen at literally any time).


> Some of the Terraform codebases I have worked on have been wildly over-complicated and take a good hour to grok. And I literally spend 10 times more of my day writing a new "standardized" Terraform module than I do writing a shell script that does the same thing.

Yep, which is why in the comment you replied to I said: "Like anything else, it is possible to write Terraform in an incomprehensible way.". You can write muck in anything.

Also, an hour sounds like a pretty reasonable amount of time to grok a totally new codebase of any significant complexity, but maybe I'm just slow. I can assure you that trying to grok the equivalent mess in a language and style you have never worked in before isn't going to be any faster.

> And Terraform doesn't even support declarative configuration management!!! The syntax is declarative, but it literally can not change the infrastructure to be the way you describe, if it has changed outside your state file (which can happen at literally any time).

I'm not sure what you mean by this. The first thing Terraform does before a plan or apply is refresh the state file with the real-world status of resources.


Terraform can only manage the resources that are explicitly defined in HCL files, in the exact way that they are defined (including based on a given module hierarchy). So if anything is either not mapped in an HCL file, or something changes in the files in an unexpected way, Terraform may refuse to do anything at all. And often this is based on computed values, meaning you don't know it's going to break until you apply, and then you have half-broken infrastructure in production.

The simplest is if you start renaming modules or moving resources in and out of different modules. Terraform will get confused and try to destroy everything rather than modify in place, even though it's the same resource, or it will sometimes just fail altogether, like it's trying to resolve some module that used to exist but no longer exists. Basically it doesn't comprehend that its logical resources are really real-world things, and leans so heavily on its logical mapping (based on things like module inheritance, which is a Terraform logic thing, not a real-world AWS resource thing) that it often just becomes unusable, and you have to perform heroics of moving about pieces of code and importing various things to work around it, if it works at all.

The bigger problem is when you have a resource which might be associated with another resource, like the various ways to represent IAM policies and roles in terraform resources. You can create the resource one way and deploy it, and then maybe someone modifies the existing real-world resource in a way that now depends on some other resource... but that resource isn't in a Terraform file. Terraform doesn't know what to do, so it will either clobber the modified state, or just die because it doesn't know how to resolve the conflict.

I would need to go back and curate a list of all the times that Terraform has just failed to do anything because something changed that it didn't expect, but basically it refuses to "fix" things that are unexpected. That, and the lack of automatic importing of existing resources is just absurd.... If Terraformer can make it work, Terraform could have too.


I love Pulumi. It's a joy to use a proper programming language to manage infra.

I wish more tools would adapt this. I find it mind boggling that it is very hard to use ansible modules in a Python program for example. That would make things much easier to compose.


We switched to Pulumi from terraform for a new project because we can use it for both cloud infrastructure and for k8s configuration. It is easy enough to just apply k8s configuration files you have, but it is harder to delete the k8s resources in all places when you delete a k8s configuration file.

Operationally they can be thought of as working in exactly the same way: they compare desired configuration, already applied configuration, and observed configuration.


> We switched to Pulumi from terraform for a new project because we can use it for both cloud infrastructure and for k8s configuration.

Did you think about using the kubernetes provider in Terraform to manage K8S as well?


Yes. Pulumi supports using existing yaml and helm templates (we aren't really using this yet), so I think it's support is better. But it may have been more influenced by the thought of having to deal with k8s boilerplate configuration in terraform's HCL. The team wanted something that was "easier to template', but I didn't want to use any template languages.


Do you also build container images through Pulumi, or do you do that separately through your CI system? Also, do you use Pulumi in your developer inner loop, or something else?


We build images in CI and reference that in Pulumi. Not sure what your question is, we use this to deploy an infrastructure environment that includes k8s.


How does it stack up to Terraform?


As a first impression it is more user friendly. The support for working with secrets is nice, but on the other hand it doesn't come free with locking support.

We need more time to evaluate the stability.


I've been looking at this for awhile and it's super cool! As a person who's used Terraform extensively and has ran into a lot of issues with it's declarative, non-scriptability (at least before 0.12), pulumi seems great. And it's also great its able to wrap any Terraform-compatible provider out of the box.

One question I have for anyone who's used it or the team if they're here is how they convinced their company to use it? Last time I checked pulumi was a small company with only 20M in funding versus hashicorp which was more established. What happens if the pulumi team runs out of money and we've moved all our infra onto pulumi?


Pulumi is great, but I gave up on persuading a company with a lot of existing terraform to move to yet another tool, and created a (pretty basic) DSL/library for writing terraform in python/c#, which can output to terraform. I could probably get permission to open source it if people are interested. *

As far as I can tell, you get more or less the same benefits as you would if you use pulumi (in the same ballpark).

* There are one or two tools for python/ruby that do this already, but having it in something with stronger typing has advantatages


I have found that the best way to use terraform is to let it be declarative and generate what you need as json (from some api or tooling).

Much less issues this way, and the json is actually what will happen with resources at a given provider. Once you start to use imperative stuff you lose a bit of predictability - your entire configuration is no longer declared, it just kinda looks that way.


Hashicorp is a bit more than just Terraform. 20m sounds plentiful for what pulumi does


As with any new technology, I'd start using it in new non-critical projects as risk is low and it makes it easy to convince the stakeholders.

To clarify, Pulumi can coexist with Terraform and other tools in the same stack, see https://www.pulumi.com/docs/guides/adopting/#coexistence


Terraform should be thought of as an intermediate representation / compilation target, right? Let Terraform take care of actually interacting with the providers, and decouple that concern from how you decide what the infrastructure should be.


I deal with a 40k line Terraform codebase. I love the idea of a more expressive language for infrastructure (and unit tests, validations, etc.), and even wrote a tiny JS library a long time ago for generating TF 0.11 from JS [1]

However, the migration paths laid out in the Pulumi docs are completely intractable for a codebase this size [2]. I'd love to give Pulumi a try, but it would only work if you could auto-generate a corresponding JS project structure from the various .tf modules and .tfstate files. Right now, it looks like tf2pulumi just spits out a single stream to stdout, which won't work for large projects like mine.

If anyone from Pulumi is here, are there any plans to make tf2pulumi a more "first-class" migration method?

[1] https://github.com/mayanklahiri/node-genesis-device

[2] https://www.pulumi.com/docs/guides/adopting/from_terraform/


We’ve worked with several Pulumi users who have migrated thousands of resources (including in production) from being managed by Terraform to being managed by Pulumi - so this is definitely possible.

We’re actively working on an overhaul of the tf2pulumi tool to support more Pulumi languages and more breath of Terraform project structures. But already it should support the majority of adoption use cases.

At-scale adoption can be done in Pulumi by combining transformations [1] and imports [2]. Together, these allow programmatically importing existing resources without the need for manual steps.

See the section on adopting in the docs here for details: https://github.com/pulumi/tf2pulumi/blob/master/README.md#us....

If you have more detailed questions on this - feel free to reach out on the Pulumi Community Slack (I’m @luke) for a deeper discussion: https://slack.pulumi.com/

[1] https://www.pulumi.com/docs/intro/concepts/programming-model... [2] https://www.pulumi.com/docs/intro/concepts/programming-model...


I'll be interviewing one of the core dev of Pulumi tonight if you're interested, he'll be doing a demo too and taking questions https://youtu.be/Sy_4KueoTUA


I welcome a new tool that competes with Terraform, but I have 2 negative points:

- where's the option to deploy to self hosted infrastructure? (E.g. Proxmox, vCenter, ESX)

- what's the "Pulumi service" and why can I self host it, but cannot deploy services to self hosted Infrastructure?


ESXi is indeed available as a target (here: https://github.com/pulumi/pulumi-vsphere).

The reasons that self-hosted infrastructure is less prevalent in tools such as Pulumi (and Terraform) are many - not least demand - but one reason is that vendors make it INCREDIBLY difficult to test thanks to proprietary license costs.

I brought up an ESXi provider for Terraform in 2015, and VMWare were not even remotely interested in giving us the ability to test it via a license grant, and I was obviously not about to pay VMWare to test integration for a product I don't even use. In the end I never released or upstreamed it, preferring to work on targets whose vendors were more co-operative.


but there's Proxmox, which is free


but the proxmox devs seem to have no interest in terraform (or pulumi) support


why not? There's a REST API and I don't think Amazon devs actively developed modules for terraform or pulumi, but the way around...so the pulumi devs seem to have no interest in Proxmox support.


I recently picked up AWS CDK and have been enjoying using it more than something like terra form.

I wonder how this compares - aside from the obvious plus of being platform agnostic (which in my case doesn’t matter)


I’ve enjoyed the CDK. I looked at Pulumi, but settled on CDK because it’s AWS developed. As where I work is an AWS shop, I didn’t see many advantages to Pulumi, and I felt more confident in AWS continuing development and support, vs another 3rd party company that could go out of business.

Also I really have enjoyed the built in unit test tooling of the CDK. Unit testing my infrastructure code has been a game changer.


Does AWS CDK handle state drift and state transitions in the same way Terraform/Pulumi does?

I believe AWS CDK uses CloudFormation under the hood which can not "intelligently" fix state drift in the same way Terraform can. CF can only tell you that tthee state has drifted, but cannot actually figure out how to fix that drift which IMO is one of the biggest strikes against CF.

The "State" section of this article has a better description of what I'm talking about: https://medium.com/@endofcake/terraform-vs-cloudformation-1d...


Ironically, having used both, Pulumi’s APIs are much cleaner/simpler than AWS CDK’s.

I feel CDK was overengineered when I last used it. Pulumi has been a breath of fresh air.


Can your DevOps read and manipulate code? If no, don't adopt a tool like this. Your Ops guys will hate you.

Furthermore, the code can't be analyzed very well with tools. (aka gradle vs. maven)

You want to make the same change to 20 scripts? Trivial with a declarative language you can parse easily.

Non-trivial if you have code. You better have those re-factoring tools at hand.

If you environment requires complex scripts to get up and running, that is a strong hint that you are not done yet. Can it be simplified further?


> You want to make the same change to 20 scripts? Trivial with a declarative language you can parse easily.

You have this exactly backwards. You don't need to make the same change to 20 scripts because you have a real programming language; reuse is built-in.

> If you environment requires complex scripts to get up and running, that is a strong hint that you are not done yet. Can it be simplified further?

What options do you have for simplifying YAML except to generate it programmatically?


If your DevOps team can’t code you don’t have a DevOps team, you have an Ops team


this is just wrong. a) it was never about ops writing code b) ops always wrote code.


The "Rule of Least Power" comes to mind: https://en.wikipedia.org/wiki/Rule_of_least_power


People forget that the "rule of least power" is "the language with the least power suitable for the task". The question is whether YAML/HCL are suitable for the task to begin with. The following facts all testify that it is not, and that the principle of least power favors "real programming languages":

* CloudFormation hacks things like references, nested-stacks, macros, imports, etc onto YAML

* HCL becomes more powerful with each version

* Helm generates YAML with templates and recently added support for Lua

* AWS is building its CDK, which generates CloudFormation with programming languages

* Things like Terraform and Pulumi exist


I've been using Pulumi for about a year for deploying stateful and stateless services on Google Cloud Platform, AMA.


Questions:

1. Have you ever used Terraform? If so, how does Pulumi compare?

2. What language do you use with Pulumi?

3. What is the learning curve like for Pulumi?

4. What specific actions do you use Pulumi for with GCP?

4a. What have been some hurdles or roadblocks you have faced using Pulumi with GCP?

5. Which Pulumi plan (cloud hosted, self hosted, etc.) and how is that working for you?


1. Yes, not as long professionally. More experience with CloudFormation and pyplates. Pulumi has libraries that wrap around Terraform providers for some of its integrations such as Google Cloud Platform.

Terraform pro:

* Simple deployments are fairly easy to grok, the base language is simple to understand

Terraform con:

* Advanced deployments, using templates, modules, functions is nontrivial to teach and error-prone.

For my use case, we have multiple environments - as most folks do - and we also have several "micro-service" like services deployed each with its own configuration. It's very easy to write a function that generates these variations:

    function someResource(args, additionalOptions, etc.) { }
This is much easier to teach, and so I would argue that Terraform solves for a case that doesn't really exist in many businesses.

2. TypeScript

3. The getting started information is fairly solid, I didn't have a lot of difficulty getting started.

4. Primarily Kubernetes resources on GKE, service accounts and tokens, Cloud SQL, storage buckets, and so on. Creating a service account, a database or storage bucket, and then linking that into a Kubernetes secret is pretty simple.

4a. I did hit a bug where Pulumi handled poorly a resource which I manually deleted in GCP. I had a Cloud SQL instance (Postgres), a User in that instance and a Database in that instance. (Cloud SQL instances can contain multiple databases). I manually deleted the three of these, and Pulumi had trouble trying to unwind state if I recall. The bug is here: https://github.com/pulumi/pulumi-gcp/issues/268

5. We aren't a paying customer - our needs around compliance mean we need to run it self hosted, and Pulumi's CLI is OSS so we are using the Pulumi CLI and some really hacky shell scripts to emulate a "server" and prevent reentrancy, effectively treating Pulumi deploys as critical sections. We have no plans to release these scripts. I understand they've recently added a self hosted option, but we haven't had time to evaluate.


Thanks. Not GCP related, but would you say Pulumi is similar to AWS CDK?

https://docs.aws.amazon.com/cdk/latest/guide/home.html


It sure looks similar. Pulumi is vendor and resource agnostic and extensible, though.

We've had luck deploying Helm charts, creating custom "resources" that are aggregations of other resources, etc.


So I've been trying out Pulumi at the new company I've just started working at and I can say the experience so far has been pretty awesome. This coming from a pre-version 1 user of terraform (when pretty much the only support was the AWS provider). For me HCL was never the killer feature of Terraform, I personally think it was the resource abstraction and providers that made it the best choice at the time. Chef/Ansible runs were always dangerous because as most of us will know from hard learnt lessons, "what you see isn't necessarily what you get". Having a DAG, tracking dependencies between resources, and getting a "this is what you have, this is what you'll get" when you apply a change is the core of what makes terraform great... but this isn't the language, I'd actually argue HCL is where most of it's problems were. The hacks one had to make before HCL 2.0 to get any type of reuse were to be frank, awful. They were also non-obvious, so you had to either pass the knowledge on in your team like a "sage" which isn't scalable for smaller/midsize teams, or let people hit those walls and waste more collective time. Writing languages is hard, and it took Hashicorp years to get it "mostly" right. Though the ergonomics of HCL I would still say aren't very nice or very developer friendly.

Now to Pulumi's merits, if you dive into it's architecture you'll find it's actually users from terraform providers under the hood (very well tested code bases). So it will do things like build a DAG, track dependencies, and give you a diff before you apply anything. From what I can tell it's just changed how you interact with the terraform internals (I'm sure it's more complicated than this, but this is my simple understanding of it). Sure you can get code ordering problems, like any imperative language but it doesn't make it any more unsafe to run as you're protected by the same plan/apply type setup as Terraform. You can also rely on the same language specific quality controls you already rely on. Things like unit tests, integration tests, code coverage, IDE integration etc. Personally I think a better use of time is teaching team members about the infrastructure fundamentals who have an interest in learning them, over getting them to also split focus on a language they will only use in one situation.


Since Pulumi extends to the k8s configuration world as well - i wonder what the Pulumi devs think of jsonnet and cue (previous discussion here - https://news.ycombinator.com/item?id=20362951 and https://news.ycombinator.com/item?id=20847943)


This post has some thoughts on ksonnet compared to Pulumi (and generally applies to comparison with the variety of other similar tools in that space): https://www.pulumi.com/blog/if-you-liked-ksonnet-youll-love-...


How is the support for AWS ECS? I really like terraform, but it is pretty bad at handling changes made to the task definition outside of it. Most of it is due to the declarative nature of the thing (You cant at the same time fetch the latest resource and create one if it doesn't exist). So the hacks we have to do to make it work with CD are just plain bad. I am considering moving our stack to pulumi (3k lines of terraform).


I'm new to this so stupid question ahead.

The thing I don't like about Pulumi (or Terraform) is saving a copy of the state (locally or on a server). This seems finicky. Isn't the current state right there in the cloud, available through APIs? Why keep a "propriotary" copy? It seems like a "what could possibly go wrong" kind of design choice.


The Pulumi/Terraform state contains more than a local copy of the last known actual state of your resources. It also contains information about the desired state is, which parts of the last known state were mutated by Pulumi/Terraform itself as opposed to external/unknown changes, etc. They are complementary.


Relying on that extra information you talk about sound like a liability more than anything else to me.

Why is there a need for anything else than a) the desired state (in the code) and b) the current state in the cloud?

The need for any extra state beyond that makes me dubious about the approach, fills me with fear and doubt about whether I really understand what the tool is doing, and so on. (I'm sure I can be wrong about this -- I'm just describing where I come from, and what I am seeking an answer to.)

Complexity always comes from state, why does one not try to have as little of it as possible...


You are correct that state is hard to manage, and consequently, the less state you have to manage, the better.

The developers of Terraform and Pulumi know this, but chose to make their tools stateful anyway. That's because if they were stateless, while they would certainly be less of a pain to manage, they would also not be as useful.

For example, without state, the tool wouldn't know what resources it has provisioned in the past. Consequently, it also wouldn't know what resources it can safely remove or modify.

Another example is that, without state, the tool cannot detect when a resource needs to be re-created from scratch, and when it simply needs to be renamed because the operator made a typo in the previous run.

In summary, yes state is hard, but sometimes you just need it to get the job done.


I really like the idea of Pulumi but I haven't really used it. To be honest, I'm not in an environment where I need to do this. A couple of scripts do the trick just as fast when combined with some clicking through.

Some things that annoyed me about TF and made me look at Pulumi actually were:

- Doing anything with loops in TF is a nightmare, and especially if you have to do things like sharding. We had a sharded KV store that was sort of optimized for our environment at my previous job and the monstrosity that you have with `count.idx` and whatnot is just ugh.

- There's a lot of repetition. Often, even with module-use you end up copy-pasting a bunch of code.


I like it, but anybody I pitched it to was like: "Ewww, TypeScript for Infra. Thanks, but no thanks!" It's using declarative infrastructure as imperative. I recently found AsCode [0], which is still Terraform, but maybe better. Pulumi for months used deprecated Azure stuff, and they didn't fix it. So, you are always late to the game, unlike when using plain Terraform. Oh, and you also need an account with them.

[0]: https://ascode.run/


May be too late for you, but you don't need an account and you can use terraform providers for anything that isn't currently supported.


I have not used Pulumi yet, but if anyone has, I'd be curious to hear a major thing re declarative/procedural (and I'll preface this with that I'm biased towards declarative for IaaC use-cases):

How does it differ from terraform when it comes to modifications and teardowns?

In terraform, I can target a module or resource for destruction, which will tear down the resources targeted and nothing else - esentially the inverse of the application resulting in creation.

How does this work in pulumi?


You can `pulumi destroy --target <urn>` in the same way as you can target a Terraform destroy.

An important thing that is not as well understood about Pulumi as it should be is that it does in fact build a declarative model, but using an imperative language instead of a DSL. Consequently there is not a huge range of difference in what can achieved from the declarative model.

Disclaimer: I was a Terraform maintainer for a long time, and also contribute to Pulumi.


I think we reached a full cycle (again), from turning complete configuration languages also known as programming languages to plain text files and everything in between.


Pulumi is awesome as a concept. It worked really well when I used it with Python. I wish they supported Ruby and had better docs for their Python client.


How does it handle idempotence?


Basically by having promises which can resolve to existing resources or new resources.


This is awesome, can't wait to try it out. We built a bunch of scripts on the AWS SDK to effectively do this - create new resources while querying for and editing existing ones. It would be nice to see more detailed docs for APIGWv2 and if there is a way to automate setup/management of both HTTP and WS instances of API Gateway - this is the closest we get to configuration nightmare.


I used to love Pulumi - the experience of managing infra + k8s resources with Typescript's type system is really nice.

However, I've found their support responsiveness via Slack / GH issues to be poor enough to where I'm strongly considering removing it from our team's internal tooling setup.


CTO at Pulumi here. Sorry you didn't get the support you needed! Please do drop me a DM in the Pulumi slack or email me at luke@.


Switched from nixops to pulumi, found it much easier to use.


This sounds like a compiler! Front-end which is some programming language, and backend targets being Terraform / CloudFormation / etc.


It's no accident that many of the original team were compiler or CLR engineers ;-)


I'm absolutely in love with pulumi. Recommended it to a CTO I was interviewing with literally an hour ago!


It’s all still a little abstract to me but it looks promising.


I'd really like to try using this from ClojureScript.


My current fave - Pulumi + JS + Ramda


Really wish they would take reasonml seriously


We avoid infrastructure deployment codes that needs to store the state. I know it's easier and faster do that but it feels so unsafe that it has the ability to delete things. We don't grant delete permissions to our automation and do deletions by hand.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: