Do you recommend any good alternative when your shell program gets too large?
Honest question, as I’m struggling to leave the shell environment once the program gets too large. I could use Perl, but $? and the likes get quickly out of hand. Python’s support for pipes was difficult last time I used it, but that may have changed. What would you recommend?
If it's too large, then just write normal Python code. It'll be a lot longer than the equivalent shell-like script, but you'll gain it back in maintenance effort, debugabillity, and robustness.
You've some hesitation with Perl, but if you stick at it, you'll find what you seek. It feels very 'unixy' and can achieve much the same as shell while being more consistent in its syntax. Its portability means it will work the same across environments. Plus the newest editions have niceties like modern classes and try/catch as inbuilt language features.
Sharing this because its the route I went, anything I'd have written in Bash I'd now do in Perl.
Thank you for encouraging me to use Perl. After Perl 6 came out I got confused at what and how to use and hence I’ve abandoned that path. I’ll try once more now.
The tooling around Perl has also gotten better over the last decade or so while also allowing you to pack everything to run on even ancient machines running old Perl 5.
Nim is statically typed and (generally) native-compiled, but it has very low ceremony ergonomics and a powerful compile-time macro/template system as well as user-defined operators (e.g., you can use `+-` to make a constructor for uncertain values so that `9 +- 2` builds a typed object as in https://github.com/SciNim/Measuremancer .
My use case is approx. like this: I can get 80% what I want with ls … | sed … | grep -v … but then it gets complicated in the script and I’d like to replace the sed or grep part with some program.
This sounds like a job for what standard C calls "popen". You can do
import posix; for line in popen("ls", "r").lines: echo line
in Nim, though you obviously need to replace `echo line` with other desired processing and learn how to do that.
You might also want to consider `rp` which is a program generator-compiler-runner along the lines of `awk` but with all the code just Nim snippets interpolated into a program template: https://github.com/c-blake/bu/blob/main/doc/rp.md . E.g.:
$ ls -l | rp -pimport\ stats -bvar\ r:RunningStat -wnf\>4 r.push\ 4.f -eecho\ r
RunningStat(
number of probes: 26
max: 31303.0
min: 23.0
sum: 84738.0
mean: 3259.153846153846
std deviation: 6393.116633069013
)
Depending upon how balanced work is on either side of the pipe, you usually can even get parallel speed-up on multicore with almost no work. For example, there is no need to use quote-escaped CSV parsing libraries when you just read from a popen()d translator program producing an easier format: https://github.com/c-blake/nio/blob/main/utils/c2tsv.nim
Good ergonomics for Perl-style quick and dirty text processing were part of the original design goals for Ruby. Those parts of the language are still there. You can write code that feels more concise than Python yet, IMO, tends to be more readable/maintainable than Perl can stereotypically be. Modern style guides, however, de-emphasize that style of Ruby since it might not be the most appropriate in the context of say a large Rails project.
Python to me, is too far away from shell/unix. It is a programming language for writing applications. For the use case of writing shell scripts but in a more powerful language, perl is still the king here (or it should be. Sadly it doesn't appear to be the case. No one is using it except for die hard gray beards.)
Raku is a modern (still a big) language with kitchen sink. Again doesn't appear to be much uptake.
Unpopular opinion, but I used Haskell "scripts" with relative success for a while. Stack has a nice script interpreter mode that is runnable in the familiar #! way.
Even allows to add dependencies and if necessary compile the script on the fly.
I once read a HN thread that recommended Go for this, and it made me interested. I think it was a useful suggestion, it made me learn Go, and I kind of agree with it, 5+ years after. It is not a shell, but it is simple and fast and useful.
Honest question, as I’m struggling to leave the shell environment once the program gets too large. I could use Perl, but $? and the likes get quickly out of hand. Python’s support for pipes was difficult last time I used it, but that may have changed. What would you recommend?