This seems great for language/library knowledge. As an experienced polyglot, the languages are not where I'm hitting the wall these days though. It's the tooling. I can learn new language basics faster than I can figure out the ecosystem/tooling.
For example, my current conundrum is how to deploy an Elixir Phoenix/MQTT app. Writing the app was a fun curve to climb. And I could use techniques like described here to learn from others in the actual programming. But how to build an executable I can wrap in a systemd process running on a different machine? Those are actions people do, not expressed so much in code I can look at. The few blogs I can find on the subject are mired in deep CI toolchains.
I want the blog that discusses the secret sauce to learn to acquire the knowledge to work the raft of ever evolving tools we have to work with now days. The "materials" (the languages) are the easy part now days. It's the massively automated complicated machinery we've built around the language of ideas that are my personal pain point of entry.
I totally agree with you about tooling being the major hurdle, and I would like to also note your particular case: deploying Elixir/Erlang the first-time is a real motherfucker.
I've deployed (server-side) code in Ruby, Python, PHP, Java, .NET, JavaScript, and probably a couple others I forgot about... but among all of those Erlang/Elixir was by far the most difficult first-time deploy (as an OTP application). It has gotten infinitely better, but probably only for those of us who have been doing it for a while (6+ years doing Elixir professionally for me). For your particular case, let me see if I can help you out ('cuz I've definitely been there).
with the "secret sauce" being to setup a build server where you will build the production release. You'll want the same flavor/version of Linux you plan on deploying to, and then copy the build artifact (tarball) from your build server to your application server (or somewhere else in-between).
One other thing to note, there's a good chance (because everyone does this) that you'll have some broken environment variables, or module attributes, because you thought they were set at runtime but they are in-fact set at compile time.
Maybe I should write that blog you're looking for...
Broken env vars had me frustrated for a few days. I've found finding the right docs in elixir to be difficult because they tend to be fairly terse and assume you know where to put things in your application, if that makes sense. They give you the function call but not explain how to parse its results.
>But how to build an executable I can wrap in a systemd process running on a different machine?
by far the most frustrating thing when I started out programming learning python was the point at which I'd made something I wanted to show my friends and I got to the 'How do I produce a standalone executable' point.
Not only an issue with advanced deployment, even for very mainstream languages the tooling story is still often not great. When I taught a CS intro class in uni this kind of stuff dominated student questions.
The trick for tooling is usually to find the blog post where the newest major version of the tooling was announced. Such a blog post will almost always contain a demonstration of what idiomatic use of the tooling looks like now. (Because, if it didn’t, how would anyone get started using it?)
This is also the domain of the more extensive language tutorials and/or “Learn X” books. Elixir’s website’s getting-started docs have a very good section on using Mix, for example, including `mix release`.
(People tend to forget to re-check an ecosystem’s official getting-started docs as new tools are introduced into the ecosystem. I’d encourage everyone to give your favourite language’s docs a quick skim every year or two; something new might jump out at you!)
For Elixir specifically, I found this post from my friend Matt to be a big help when I was deploying my first app. The Phoenix docs section on deployment were also useful.
The time I've spent on the Github actions is substantively higher than the time I've spent on the .rs files. Of course you can't "test actions before commit" in the way you can actual code, so I kept having to make branches, make 15 commits like "try action fix again", followed by squashing them all down and merging.
That part where you have to make a commit in order to attempt to fix a CI problem was driving me nuts. It‘s the same with Travis.
Concourse gets this right - you can run a pipeline task as a one-off from your workstation until it’s done, and only then check it in. And even ssh into the build container in order to debug build failures.
As someone who codes stuff by myself a lot and needs to pick up tools I've noticed a lot of the knowledge feels like it might come through oral tradition. Handed from developer to developer.
Places have learned which way through the tools cause the least problems and will have an informal community of practice around tools - where you can just quickly ask someone something that may have seemed too trivial to the tool creator to put in the docs.
1) find out if the runtime/framework is supported but Heroku or if there are any buildpacks available.
2) spin up a Dokku instance using Vagrant for local development and testing
3) deploy to a live Dokku server
If/when I encounter any issues I add Heroku or Dokku to my search query and 9 times out of 10 I’ll find an answer to my issue. Else I just dig into the Dokku docs and GitHub issues and figure it out.
So for instance googling for deploying a Phoenix app with Dokku results in a few hits such as this one [0].
There's also a lovely UI for Dokku being actively developed: Ledokku [1]
The only drawback currently is when you want to horizontally scale your deployment. You can use their kubernetes or nomad schedulers but I think those are an overkill in terms of complexity. You could also use a load balancer in front of multiple Dokku instances but you then lose the ease of deployment, configuration, etc…
Which is why I think their docker swarm scheduler [2] will be one of the most important feature they could add. It’s currently on the roadmap but I’m sure with a bit of sponsorship and a few pull/merge requests it will become a reality.
> But how to build an executable I can wrap in a systemd process running on a different machine? Those are actions people do, not expressed so much in code I can look at.
The former sounds like a makefile, and the latter sounds like a Terraform plan (perhaps combined with something like Kubernetes manifests, but that’s getting more architecture-specific). These days I don’t think there’s any excuse to use the point-and-click approach for setting up infrastructure: it’s effortful, bug-prone, a security hazard, means everyone has to be trained in yet another area, and risks accidentally spending far more money than you intend (either by using surprisingly expensive services like Spanner, or by inadvertently leaving unused infrastructure running).
That said, I do agree that platforms like AWS are unnecessarily complex for the vast majority of CRUD web developers. The complexity makes sense for the small percentage of people who are genuinely setting up a very idiosyncratic and unique architecture, but the 98% of CRUD developers really need an opinionated platform, perhaps built on top of AWS/GCP/Azure and modelled on v1 platforms like Heroku, which would set up the infrastructure you need for the average web backend.
> The former sounds like a makefile, and the latter sounds like a Terraform plan (perhaps combined with something like Kubernetes manifests, but that’s getting more architecture-specific).
Were you trying to further illustrate the tooling point?
Made multiple contributions to the CairoGraphics project back in the day. Biggest problem? The insane "clever" Makefile structure one of the maintainers had set up. It worked as long as it worked. If it needed to change, one guy alone pretty much was able to tune/change it. It was a language unto itself.
Sorry, I was responding to the “not expressed so much in code I can look at” point. All these things should be expressed in code nowadays. I’m sure some Makefiles are excessively complex - and Terraform and K8s manifests surely as well - but you should hopefully be able to hunt down some good ones.
If you find some project that do this right and use tools you need, just ask the author to publish his knowledge. It will be beneficial to everyone not only you. Most people like to share their knowledge, and if you find that other person, just try to find another one.
A year ago I've read article about the person that self-published a book (printed and ebook), here wrote an article how he do that and how he created it in markdown (very well written book about TypeScript in Polish), I've asked an year ago (when read that article) if can publish the code, he said that he was thinking about this. Recently he published how own blog system online on GitHub and wrote another article this time with link to GitHub. I still waiting for the book code.
Hit it in a nutshell. I find C++ great as a language - complex but wonderfully powerful. It's like wielding a sharp [DragonSword]. However, the tooling is in the stone age compared to Rust. And there doesn't seem much cross-vendor interest in improving matters. The committee sticks to the language and not the broader ecosystem.
This makes it a no-go for younger programmers who are spoilt by friendly tooling - great build systems, package managers, superb documentation tool compared to the ancient, decrepit dinosaur that is doxygen, etc
For example, my current conundrum is how to deploy an Elixir Phoenix/MQTT app. Writing the app was a fun curve to climb. And I could use techniques like described here to learn from others in the actual programming. But how to build an executable I can wrap in a systemd process running on a different machine? Those are actions people do, not expressed so much in code I can look at. The few blogs I can find on the subject are mired in deep CI toolchains.
I want the blog that discusses the secret sauce to learn to acquire the knowledge to work the raft of ever evolving tools we have to work with now days. The "materials" (the languages) are the easy part now days. It's the massively automated complicated machinery we've built around the language of ideas that are my personal pain point of entry.