
Dgsh – Directed Graph Shell - pcr910303
https://www2.dmst.aueb.gr/dds/sw/dgsh/
======
j88439h84
Python's mario is another terminal-oriented tool with concurrent task
execution, but with async concurrency rather than multiprocess parallelism.

For example, to get a bunch of urls concurrently

$ mario async-map 'await asks.get ! x.text' < urls.txt

[https://github.com/python-mario/mario](https://github.com/python-mario/mario)

~~~
IggleSniggle
Was mario inspired by spotify's luigi? Coincidence?

[https://github.com/spotify/luigi](https://github.com/spotify/luigi)

------
dang
A thread from 2017:
[https://news.ycombinator.com/item?id=13352659](https://news.ycombinator.com/item?id=13352659)

~~~
IggleSniggle
Thanks for posting that. There is good additional discussion there.

Edit: SciPipe looked particularly interesting.
[https://github.com/scipipe/scipipe](https://github.com/scipipe/scipipe)

------
Legogris
Thanks for sharing - I was previously pondering a bit how a generic graph pipe
shell syntax would look - I might try this out myself soon :)

------
keid
1980, operating systems class, write a shell for UNIX. I thought I'd be clever
and do more than simple pipes, so I created some syntax to enable the
specification of a directed graph. It's pretty neat to see that someone has
now done this industrial strength.

~~~
kedean
I think it's a pretty natural idea. According to Brian Kernighan's memoir, the
original vision for pipes was more like a mesh than the omnidirectional flow
we ended up with:

"Doug wanted to allow arbitrary connections in a sort of mesh of programs, but
it was not obvious how to describe an unconstrained graph in a natural way,
and there were semantic problems as well: data that flowed between programs
would have to be queued properly, and queues could explode in an anarchic
connection of programs. And Ken couldn't think of any real applications
anyway."

My first thought when reading over the docs for dgsh is always something
similar...what's the real application? What does it gain me other than some
mathematical elegance that I don't lose in harder to read scripts? The
examples all pretty much collapse to doing things in parallel and collecting
them via cat, which seems a little basic for something as abstract as a DAG.

------
elmolino89
Interesting effort. On the other hand there is a bunch of other more popular
and battle tested tools from Apache Airflow to a bioinformatics pipelines
managers, such as Nextflow, Snakemake etc.

------
sdan
Looks pretty interesting (automatically deleting duplicate files). Bookmarking
to see if I can utilize this!

------
SeriousM
Dgsh sounds like dogshit... Sorry

~~~
sdegutis
Weird, my first thought when reading it was a character from Morrowind...

