
Using GNU parallel (YouTube tutorials) - uranium235
http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
======
stephenr
We've briefly used parallel for an internal tool when the site went
"multisite" (i.e. multiple parallel instances that sometimes need to all do
the same thing, such as post-deployment tasks).

We currently have a patch to remove it, and instead use regular task
backgrounding and the `wait` shell builtin.

The way output is buffered was not particularly conducive to what we wanted to
achieve.

~~~
MachineMartin
THE way? It sounds as if GNU Parallel only buffers in one way.

GNU Parallel can handle output in a few ways:

    
    
      * --ungroup: With -u output GNU Parallel does not touch the output and output may mix
      * --line-buffer: This buffers a single line from each process.
      * --group: This buffers in files in $TMPDIR. $TMPDIR can point a RAM disk, if you want to buffer in RAM.
      * --sqlworker: This buffers in memory before saving output to an SQL server.
    

If you are using '&' \+ 'wait', it pretty much has the same effect as
`--ungroup`.

