On: Ruby: A great language for shell scripts

[mikepurvis]

Pex was also the solution I landed on after evaluating several non-container options for distributing a Python project to arbitrary Linux hosts.

It works well but with one huge caveat: although you bring the stuff required to reconstitute the venv with you, you’re actually still using the system’s python executable and stdlib!! So for example if you want to make a project targeting all supported Ubuntu LTS versions, you have to include the wheels for every possible python version you might hit.

Ultimately this boils down to there not really being a story for statically compiled python, so in most normal cases you end up wanting a chroot and at that point you’re in a container anyway.

[danmur → mikepurvis]

Nuitka has worked for me for everything Ive tried (in house dev tools). I didnt end up using it for work because I can rely on a pristine system Python with the right version so pex makes more sense.

There are other options I didnt look too much into, e.g. Beeware

[networked → mikepurvis]

I wish an easy cross-platform PEX or shiv [1] were a thing. Binary dependencies are the biggest reason I prefer the new inline script metadata spec (https://packaging.python.org/en/latest/specifications/inline...) and `pipx run`. Luckily, they're pretty great. They have changed how I write Python scripts.

The way inline script metadata works is that your script declares arbitrary dependencies in a structured top comment, and a compliant script runner must provide them. Here is an example from a real script:

  #! /usr/bin/env -S pipx run
  # /// script
  # dependencies = [
  #   "click==8.*",
  #   "Jinja2==3.*",
  #   "tomli==2.*",
  # ]
  # requires-python = ">=3.8"
  # ///

pipx implements the spec with cached per-script virtual environments. It will download the dependencies, create a venv for your script, and install the dependencies in the venv the first time you invoke the script. The idea isn't new: you could do more or less the same with https://github.com/PyAr/fades (2014) and https://github.com/jaraco/pip-run (2015). However, I only adopted it after I saw https://peps.python.org/pep-0722/, which PEP 723 replaced and became the current standard. It is nice to have it standardized and part of pipx.

For really arbitrary hosts with no guarantee of recent pipx, there is https://pip.wtf and my venv version https://github.com/dbohdan/pip-wtenv. Personally, I'd go with `pipx run` instead whenever possible.

[1] I recommend shiv over PEX for pure-Python dependencies because shiv builds faster. Have a look at https://shiv.readthedocs.io/en/stable/history.html.

[mikepurvis → networked]

All else being equal I’d probably prefer poetry for the broader project structure, but that would definitely be compelling single script use cases.

[networked → mikepurvis]

You can also combine the two. Something I have done is script dependencies in inline script metadata and dev dependencies (Pyright and Ruff) managed by Poetry.