
Show HN: Cr – Runs your tasks at maximum concurrency - cirowrc
https://github.com/cirocosta/cr
======
znpy
That's what make has been doing for the last like 40 years.

~~~
blablabla123
I wish the make documentation was better though. I've also read oftentimes
that people say that turning on make concurrency with `-j` is on your own
risk.

~~~
xyzzy_plugh
That's because people write shitty makefiles. Most makefiles I see are full of
improperly used double colon rules, phony targets not marked .PHONY and
implicit ordering assumptions.

If `-j' was enabled by default, then it would be much harder for writers to
develop these issues. But I suspect most of these authors, who are already
unwilling to learn make properly would simply give up and use a different
tool.

~~~
f1notformula1
> That's because people write shitty makefiles.

Which, coincidentally, is my main complaint with Make: It's way too easy to
write shitty Makefiles. There are way too many wrong ways to do things.
Arguably this is a community problem and not a tool problem, but it doesn't
change the fact that most Makefiles are written badly.

I used Make professionally for 5 years and have since moved to other tech
stacks that don't require Make but have tools that are far worse in every
other way ( _cough_ Ant _cough_ ). But IMO the reason they win out is they are
far less flexible, so there are fewer ways to do things, so they work better
in aggregate.

Build tooling is something that most devs look at very rarely, so they never
build expertise. Ergo, build tooling shouldn't require expertise.

If my theory holds out, I feel Gradle will become the new Make in a few years
or a decade - very powerful but too many wrong ways to do things.

~~~
xyzzy_plugh
Make is more than 40 years old -- 40 fucking years! And we still use it. It's
everywhere. It's not going anywhere. Gradle will not replace it.

Why? Because make works. Because it's actually easy to understand if you've
read the manual. It's simple. People try to make it do really complicated
things, which you are better off putting in a script or another program. At
it's core, make works and does its job very, very well.

Here is a very beautiful, readable, portable makefile:
[https://github.com/git/git/blob/master/Makefile](https://github.com/git/git/blob/master/Makefile)

It's hard to write portable _anything_ without details leaking out. Go is the
best implementation I've ever seen here. It works identically everywhere I've
tried it. The approach the Go team has taken is extreme. I think that says a
lot.

Both these tools are very simple, and limit what you can do. When you try to
go outside the box, you hurt yourself. Don't hurt yourself.

Tools like Gradle make it real easy to hurt yourself without realizing it.

~~~
kazinator
We don't literally use _make_ , exactly as it was forty years ago.

> _Here is a very beautiful, readable, portable
> makefile:[https://github.com/git/git/blob/master/Makefile](https://github.com/git/git/blob/master/Makefile)
> _

That's a GNU Make Makefile. It's portable to installations where you have GNU
Make ported already.

~~~
Spivak
Look, this battle has been lost for ages. People only seem to write portable
Makefiles by accident and usually just target the implementation of make on
the platform they're developing on.

------
jchw
Everyone is going to compare it to Make, but it's not like Make syntax is
ideal or consistent; I can see this being especially useful when using it in
part of a pipeline, where maybe the YAML is generated elsewhere.

~~~
garmaine
> but it's not like Make syntax is ideal or consistent

It's standardized in POSIX.

"Ideal" is subjective.

~~~
jchw
Fine. But most sane people would agree that Makefile generation is not a fun
business. Autotools, anyone?

OTOH, generating YAML documents is easy. I already write code that generates
YAML documents.

P.S.: POSIX make doesn't support concurrency. So, it's not fair to compare
this purely to POSIX make.

~~~
sirn
> OTOH, generating YAML documents is easy.

On the topic of syntax, I'll say YAML is not the best format for configuration
either. At a glance, it looks simple and easy to write, but YAML can bite its
user in an unexpected ways due to its implicit typing[1]. For example this
YAML:

    
    
        on: yes
    

…will be parsed to:

    
    
        {true => true}
    

…which cannot be serialized to JSON in a lossless way (since JSON spec specify
that a key must be a string):

    
    
        {"true":true}
    

…and translating this back to YAML would result in:

    
    
        'true': true
    

This is without considering other features such as anchor, aliases and
tags[2], which further make the YAML spec incredibly complex.

[1]:
[https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst...](https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst#what-
is-wrong-with-implicit-typing)

[2]:
[http://www.yaml.org/spec/1.2/spec.html](http://www.yaml.org/spec/1.2/spec.html)

~~~
icebraining
I agree that the implicit typing is dangerous, but developers shouldn't expect
to convert YAML to JSON losslessly. It's a superset, after all.

------
jlebrech
I miss read this, I thought it was an implementation of workers with a
dependency graph.

for similar projects look at:

Bazel [https://bazel.build/](https://bazel.build/)

Tup [http://gittup.org/tup/](http://gittup.org/tup/)

Make [https://www.gnu.org/software/make/](https://www.gnu.org/software/make/)

------
EdSchouten
What about Ninja? [https://ninja-build.org/](https://ninja-build.org/)

------
gabrielcsapo
This is awesome. I have been working on something similar, but added a UI to
visualize the tasks and the order they ran in.

[https://www.gabrielcsapo.com/build.sh/](https://www.gabrielcsapo.com/build.sh/)

------
stephenr
So, `make -j`?

------
gyoubu
Looks nice! Will inspire my project as well. Made something recently as well
which is based on the same principle. A simple parallel/sequential task runner
which has built in retries and allows steps to be continued even if they
return errors.

[https://github.com/abelmokadem/scriptz](https://github.com/abelmokadem/scriptz)

------
peterwwillis
I'm sure somebody has a use for this. But with no scheduler tuning, no
prioritization, no dynamic configuration, no multiprocess logic, etc it's
effectively just "xargs" with dependency tracking, written in Go. And I'm
pretty sure you can do this with Ansible.

~~~
dullgiulio
Indeed it's almost trivial. I wrote a similar tool that is also able to split
chunks of work between different machines:
[https://github.com/dullgiulio/empa/tree/master/cmd/ep](https://github.com/dullgiulio/empa/tree/master/cmd/ep)

But again, no IPC, just a simplified "GNU Parallel."

------
jitl
A co-worker of mine recently implemented something very similar.

Our major feature over Make was emitting events as newline separated JSON, so
that a downstream process can visualize them in a pretty way, track progress,
etc.

We use ours run multiple Chef recipes in parallel.

------
jdright
I thought it was related to this:

[https://github.com/fungos/cr](https://github.com/fungos/cr)

It was recently over Twitter.

------
redshirt
The nice clean dependency graph is pretty cool. Not having tabs and spaces as
an integral part of syntax, priceless.

------
yellowapple
License?

