On: GNU Parallel

[wingless]

Am I the only person who finds GNU parallel way too complicated? I tried to perform a very easy parallel task with it and spent hours reading the documentation and various tutorials. If a person with Unix command-line skills can't easily pick it up, what's the point of having it?

[tyingq → wingless]

There do seem to be some very complex use cases. On the other hand, their example of parallel gzip of files seems straightforward:

find . -name '*.html' | parallel gzip --best

Generally, using it in places where you would normally use xargs seems uncomplicated.

[amelius → tyingq]

Probably a stupid question, but how do the remote machines translate the pathnames to local pathnames? And what happens if they fail?

[tyingq → amelius]

The example above is just running in parallel locally.

[ole_tange → amelius]

A good start is looking at the section 'Remote execution' in 'man parallel_tutorial'.

Spoiler: You can have GNU Parallel kick the machines if the fail, and rerun the job on another machine.

[mynewtb → wingless]

What was it that you tried to achieve? I find it very easy for trivial parallelisation over files but am quickly lost when it becomes more complex.

[ole_tange → wingless]

Did you start by reading the "Reader's guide" which is presented before the first option is introduced in the man page?