1. has a compatibility mode with sh and bash. it's the only new shell I know with a upgrade path from bash 
2. otherwise fixes tons of unix shell misfeatures and in particular fixes quoting hell 
PS: one fun quirk of the oil implementation is that it's written in Python.. but actually in a DSL that generates statically typed C++ code, ditching the Python runtime altogether   - when you do compile Oil, you just get a C++ tarball (somewhat like fftw, a FFT package written in OCaml but which generates C code; fftw developers write OCaml, but fftw users compile C)
 But actually I think there's still work to do to remove the CPython dependency at runtime, https://github.com/oilshell/oil/issues/636
- murex https://github.com/lmorg/murex
- Elvish https://github.com/elves/elvish
- NGS https://github.com/ngs-lang/ngs
- And any language REPLs turned into shells, such as Python, LISP, Lua and others.
The author did say there is more to follow though so maybe the scope of this article was a little more closed by intention.
I didn't know about fftw, but another nice analogy is TeX by Donald Knuth! TeX is actually written in WEB / a subset of Pascal (H-Pascal I think), and then translated to C. That is, it's NOT typically compiled with a Pascal compiler (I wonder if anyone even maintains those?). This came up a few times on Hacker News and I got some interesting details.
I wrote yesterday that I want to hire a Compiler Engineer to help with the C++ translation:
It's already half done, passing ~1100 of 1950 spec tests, so I believe it's very feasible. But it needs someone who can concentrate on it full time.
I might as well post the draft of the "job listing" to get people thinking about this. I think the challenge is finding the right person rather than funding it (I'm applying for funding now)
https://github.com/oilshell/oil/wiki/Compiler-Engineer-Job (rough draft, you may see polished versions of this later)
One sentence description: write an ~10K line translator for statically typed Python, and a ~5K line C++ runtime, and get a free shell out of it! (over 5 years of painstaking work on the semantics!)
The input is ~36K lines of Python the output is ~100K lines of C++. This is a very doable project for an experienced compiler engineer. :) So if anyone knows a good candidate, send them my way at email@example.com.
Also, I (virtually) attended the panel where this paper originated:
It was basically a call for researchers, hence lesser focus on existing and new implementations.
I think it's good to publish future-facing ideas (almost all of which I've read), whereas Oil is a lot of engineering work!
FAQ about the name: http://www.oilshell.org/blog/2019/06/17.html#why-is-the-proj...
For better or worse, it's what I and my team used to manage a decent size VMware cluster of a hundred hosts and 6k VMs ( I think, it was a very long time ago)
Wouldn't go back, but there are aspects I miss
There are also other typed shells that appeal to yourself which aren't Powershell but do have behaviours similar to Powershell and which do play a lot more nicely with the wider Linux / UNIX ecosystem:
I'm the maintainer of murex so happy to answer any questions on that shell here. I do know the the maintainers of Elvish and NGS pop on HN too.
The gp (Paul) is not "advertising Powershell" and he's not recommending people switch to it.
Instead, he's saying that it's strange that an academic paper talking about future concepts doesn't even have a cursory survey of what other popular shells have done.
Here's an example of that style of writing where other platform technologies are mentioned: Joshua Bloch is a ex-Sun employee and Java advocate proposing new syntax for Automatic Resource Management and in his note, he mentions what C# and C++ did: http://mail.openjdk.java.net/pipermail/coin-dev/2009-Februar...
It shouldn't have to be said that citing C# & C++ does not mean it's an "ad for C#".
When a new 2021 paper about future of shells has no mention of zsh, Powershell, etc by the authors, it's reasonable for readers to wonder if the team has overlooked what others have done. E.g. Powershell was designed by ex-UNIX admins.
50 years is quite the time horizon after all
The title is promising, but when I read the paper half a year ago I found it disappointing, no practical value. Academic considerations to get a conference paper.
Shell 'scripting' is only supposed to be a way to automate that IDE. It's not supposed to be a programming language. Literally the purpose is that you're sitting there at your terminal and you think "boy, I would really love to rename 100 files in a certain way". At first you write a quick one-line series of commands with a loop. But the intention is to eventually write a real dedicated program (like the `rename` command) so you don't have to write a crappy line of "scripting".
Shell scripting should remain limited in functionality, and be tailored to helping someone use the IDE. If you want a systems programming language, then develop one. Go crazy and develop new ideas and push the boundaries of computer science. But please don't ruin the shell experience by trying to solve some unrelated problems with it and end up ruining the simple end user experience.
This paper reads like it was written by clueless academics with no real world experience.
Multicore bash already exists and is seldom used, through the "parallel" cmd.
Other points address no particular issue, and would actually make the language more painful to use.
Bash has its limits, which pushes you to use a full fledged language if you attempt to walk past them. It's a good system.
I wrote 75 lines of bash to do failure-resistant streaming parallel computation on heterogeneous distributed systems. "parallel" was very close to doing what I want, but it wasn't particularly hard to do it myself without it. I fire it up, peg every core on as many spot instances as AWS will give me, then shut them all down.
I regularly pass data structures through pipes, and "jq" does everything I have needed so far.
The tools can always get a bit better, but I don't need my duct tape to be any smarter, KTHX.
Bash was good enough when being a sysadmin meant your structured data was largely just flat lists of plain text. However the industry moved away from that model years ago. And these days it’s more pronounced than ever with myself spending more time in YAML, JSON and tables thanks to modern DevOps tooling than I have done in the 2 decades of experience I had prior.
Sure, jq is amazing. But if I’m having to reach for jq in nearly every pipeline then one has to question why the shell can’t just understand JSON natively itself. And why should I have to learn a different tool with their own formatting language for working with YAML, CSV, XML, S-expressions and so on. Why can’t I use the same shell primitives regardless of the underlying data format is?
This is why I switched away from bash. I’m enough of an expert in bash to use it for DevOps everyday but I’m also pragmatic enough to realise that other shells actually do offer quality of life improvements. This doesn’t mean those shells have to be fully fledged programming languages - I’m really not suggesting that we should be using shell scripting for anything complicated. The problem is the stuff that isn’t complicated has evolved to become complicated in bash because the nature of our work has shifted away from what bash did well.
But each to their own. I wouldn’t want to impose my workflow on others anymore than I’d want others to lecture me about theirs. I just wanted to offer a counterpoint from someone who is also a seasoned sysadmin (and more recently “DevOps”).
I'm curious, though, what other shells offer 'quality of life improvements' ... and can I convince my colleagues to install it everywhere?
To be honest most alt of them do. Bash has enough footguns to incapacitate a small army. Seasoned users like ourselves know how to mitigate them but using a shell that doesn’t require a conscious effort to support spaces in file names and other really basic things is one less piece of mental overhead I need to consider when working.
> and can I convince my colleagues to install it everywhere?
I’m not sure what you mean here. Are you trying to convince your colleagues to switch so you all use the same shell or saying you still spend a lot of time in remote SSH sessions?
Either way, it sounds like you’re probably best sticking with Bash if those are your requirements.
My remote workflow is largely API driven (serverless) and the few times I do need to SSH into a remote instance it’s usually just to perform basic tasks. The real meat of my console work is done locally these days.
Bash Technical Fellow?
Bash (and other shells) are very good at working with other programs and using the filesystem. It's true you have to know many idioms, but once you're there, you can be quite productive for these kinds of tasks.
Subjectivity of what is complex is the problem here, which means that regular Bash users regularly write programs that are complex to me. And I'd much rather use something with less footguns, even if the good parts of the shell become less easy.
One's level of expertise will determine what is "too complex for $SHELL". When that point is reached, it's time to switch to something else, not time to add more features to $SHELL.
(If, when something becomes too complex for $SHELL, you switch to assembly, you might be too much of a $SHELL expert!)
If the bash program evolves to span multiple files and lots of lines, I usually switch to ansible/ruby/clojure
Bash is an error-prone insane mess. It's not "a good system". Frankly it's so obviously terrible that I wouldn't trust the opinion of anyone that calls themselves a "bash expert"! It's like being a punch card expert.
Actually that's unfair to punch cards. They were well designed at the time. Bash was born deformed.
The pdksh version of the Korn shell was available to the POSIX shell standardization committee. It was not chosen because it was too large, with too many features.
Another round of less-capable tools does not seem productive.
Why is this important consideration for you? What's the use case?
The substitution and shell-quoting are the things that piss me off. Why in 2021 would ever want all that legacy cruft? I better write a Python script that is far more readable and intuitive than figuring out all the symbol combinations in shell programming. We deserve better, don't we?
Shell programming is like necromancing, one simple mistake leads to big disasters. Also, these shells never point out the actual mistake, like are very unhelpful.
FWIW, I have switched to Powershell & Python for most of my work. I only use Python when I have to share scripts with colleages, though. Powershell still beats Python for scripting.
Check out op https://docs.shoreline.io/op
op> hosts | filter(cpu_usage > 80) | `pkill high_cpu_proc`
- We already have a dataflow graph, it's called makefiles
- Going from shell commands to Ansible playbooks can be made easier
- Declarative, idempotent commands
- Integration with version control
- GNU standards. If I cannot use --help and --version, I get angry
- Backward compatibility. Ripgrep, I'm looking at you
- Autocompletion is still in its infancy. Co-operative autocompletion is easy, and can be done by utility authors. What are the options (no pun intended) when the author would not update an autocompletion script? fzf and its kin greatly enhance user experience
- Performance. Accidentally quadratic stuff is a Cthulhu-scaled abomination
- Fancier prompts without needing to install extra software or add a bunch of PS1/PS2 scripts
'Novelties' (not so much, other OSes have had them before):
- Easier parallelization
- Improvement in job control - freezing and thawing jobs, moving jobs to other hosts etc. K8s/kubeflow and friends
- Better integration with secret managers
- History cleanup when passwords are passed on the command line
Is that really that important? I do have a few POSIX incantations ingrained into my muscle memory, but it’s not a good thing imo. Is that 4 God knows what flag and strange order really that important? Maybe add a separate backward compatible mode when issued through the old tool’s name, but I would like at least a partial revamp of the frankly, not too ergonomic POSIX tools. While Powershell can sometimes be more verbose, I would prefer a similar convention.
It is a damper on gung-ho, brazen disregard for existing code/user base.
It prevents code base rot and saves money for corporations/governments.
It forces developers to adhere to the principle of least surprise, and ensures smooth user experience. Mind you, Powershell beats POSIX on readability hands down, and autocompletion would make longer, more readable options much easier to write. Still, the easiest way to increase adoption is to meticulously copy conventions from predecessors.
Compatibility mode would be enough for most purposes if it's maintained and not dropped three years later in the name of progress and limited maintainer attention span.
What specific backward incompatibilities have caused you problems?