
moreutils - JoshTriplett
https://joeyh.name/code/moreutils/
======
Koahku
This is a pretty good collection of small utilities but I would recommend
using GNU Parallel[0] instead of the included parallel. It contains a ton of
extremely useful additional features such as distributing jobs to remote
computers.

[0]:
[https://www.gnu.org/software/parallel/](https://www.gnu.org/software/parallel/)

~~~
jaytaylor
One big turnoff with GNU parallel is that the first time it is run on a
machine, you must interactively accept a EULA. There are ways around it, but
as a rule of thumb I avoid command-line tools that do this, because this
inconsistent runtime behavior is annoying _at best_ and can easily break and
complicate automation.

And no, I don't want to set an environment variable on a fleet of machines to
suppress behavior which shouldn't be enabled by default to begin with! _<
cough> .NET telemetry </cough>_

A real shame, because it is an otherwise useful tool IMHO.

~~~
telotortium
What it actually does is to beg you to cite parallel if you use it in academic
work. There's two ways to silence it:

• Run `touch ~/.parallel/will-cite`.

• Pass the flag `--will-cite`.

Still annoying, although it's easy enough to disable.

------
sigcode
I propose a new project: lessutils. You post about some program that you use
for some simple task that's not among the "standard UNIX utils". (ed, sed,
AWK, lex, etc.) Then we show you how to do the same task with only the
standard utils. (i.e. no install needed) -- inevitably someone shows us how to
do it in AWK and makes all feel stupid. :) If we are successful, you get to
eliminate one more dependency from your system, not to mention reducing attack
surface.

~~~
spinningarrow
Here's one I use: jq for JSON parsing

~~~
cben
Yes. letsencrypt.sh is a nice case study, spending a lot of effort to parse
json with essentially bash & grep. It works now but is full of assumptions
("this array of hashes will not contain [ anywhere"). There is nothing in
POSIX that's _well suited_ to manipulating json in the way that jq is.

------
mkj
"nobody thought to write long ago"

Quite a few moreutils involve unbounded buffering of input, I wonder if they
were left out of Unix originally due to memory limits.

------
jarcane
_I 'm a fan of the unix tools philosophy, but I sometimes wonder if there's
much room for new tools to be added to that toolbox. I've always wanted to
come up with my own general-purpose new unix tool._

This was something of the inspiration for wf as well.
[https://github.com/jarcane/wf](https://github.com/jarcane/wf)

~~~
gnuvince
Just yesterday, I asked on lobste.rs [1] how I could take my small utility to
compute the minimum, maximum, and expected value of a dice roll expressed in
D&D notation and make it a better Unix citizen. The commenters were helpful,
and with few changes I was able to make the program usable in a pipeline, even
if I don't really expect that's going to be a use case for me.

As an aside, my utility is also written in Rust; nice to see more of these
small programs written in Rust.

[1]
[https://lobste.rs/s/fdzyio/how_should_i_make_this_small_prog...](https://lobste.rs/s/fdzyio/how_should_i_make_this_small_programs)

------
minitech
Note that “things like this” can sometimes be done just using sed:

    
    
      % sed -i -e "s/root/toor/" -e "/joey/d" /etc/passwd

~~~
phamilton
That's platform dependent. BSD sed doesn't support -i

~~~
_delirium
I don't think that's true anymore, though it was historically. The sed
currently shipped with both FreeBSD and OpenBSD supports -i, at least.

------
stevebmark
Once you realize that awk, while useful, is really just a terrible programming
language with terrible syntax that you have to write in a string, with random
unreadable environment variables to try to hack in useful functionality, then
you are finally free.

~~~
Yuioup
Are there any modern alternatives out there?

~~~
otabdeveloper
I had to go and write one:
[http://tkatchev.bitbucket.org/tab/](http://tkatchev.bitbucket.org/tab/)

~~~
tomcam
Hey, that's nice.

------
bluejekyll
> sponge: soak up standard input and write to a file

How does this differ than just redirecting to a file in the shell?

~~~
cgag
If you do something like:

    
    
      echo "hello world" >> dummyfile
      cat dummyfile | sed 's/hello/goodbye/' > dummyfile
    

dummyfile will be blank afterwards

if you do:

    
    
      cat dummyfile | sed 's/hello/goodbye/' | sponge dummyfile
    

It'll work as you'd expect (or at least as I'd expect). Otherwise `>
dummyfile` will truncate dummyfile before it's read and inputed into sed.

~~~
StillBored
For the sed example, what you actually want is -i which modifies the given
file.

AKA

sed -i 's/hello/goodby/' dummyfile

~~~
phamilton
That's only in gnu coreutils. BSD coreutils don't support -i

~~~
cheiVia0
didn't

[https://www.freebsd.org/cgi/man.cgi?query=sed](https://www.freebsd.org/cgi/man.cgi?query=sed)

[http://man.openbsd.org/OpenBSD-
current/man1/sed.1](http://man.openbsd.org/OpenBSD-current/man1/sed.1)

[http://netbsd.gw.com/cgi-bin/man-cgi?sed++NetBSD-
current](http://netbsd.gw.com/cgi-bin/man-cgi?sed++NetBSD-current)

------
smegel
jq would be a nice addition to that list.

~~~
matt_wulfeck
Are there any tools in moreutils that aren't written in portable C? i agree.
Jq is a must.

~~~
liw
I doubt joeyh wants to add tools to moreutils that already exist as separate
projects of their own.

------
nickysielicki
I can vouch for Vipe, I have found it immensely useful at times when throwing
awk at my input turns out to not be as straightforward as I had hoped.

------
wtracy
A more detailed post on the subject:

[https://rentes.github.io/unix/utilities/2015/07/27/moreutils...](https://rentes.github.io/unix/utilities/2015/07/27/moreutils-
package/)

------
Zhycrin
Good utility, but I would reccomend using the GNU ones as mentioned by another
user although it will infloop the proccessess, you can use command-line tools
to go around this.

------
perlgeek
lckdo looks a bit similar to flock(1), though knowing Joey from reputation,
I'm pretty sure he had good reasons for including lckdo.

~~~
mrob
From the lckdo manual: "Now that util-linux contains a similar command named
flock, lckdo is deprecated, and will be removed from some future version of
moreutils."

------
dfc
vidir and vipe are two of my favorite commands from moreutils.

~~~
njt
Also check out combine. It allows you to combine lines from two files using
boolean operations:

    
    
      combine file1 [OPERATION] file2
    

Where [OPERATION] is one of [and, not, or, xor]. It allows you to quickly pull
out interesting data from your files (the input files don't need to be
sorted).

------
throwaway_exer
ts is very useful for timestamping unbuffered output to do simple profiling on
production systems.

