
Show HN: Test your JavaScript modules simultaneously in 32 different versions of Node.js - diggan
http://victorbjelkholm.github.io/autochecker/
======
diggan
This is a utility I build to be able to make sure my JS CLI's actually work
across the version I want to support.

Right now, the tool also works across any language where there is docker
images with versions, not just for JavaScript/NodeJS. It's enough with having
a Dockerfile with $VERSION (that will be replaced by the tool) and changing
the test command. I will make this easier in the future.

Thanks for taking a look and please let me know what you think about it.

~~~
caffinatedmonk
Just out of curiosity, can't this be implemented with `nvm` and some sand
boxing? Why did you choose to use docker?

~~~
diggan
Yes, it can indeed. I have three reasons for not doing it though.

The first is that nvm is nodejs specific and since I knew I wanted it to
eventually work with multiple languages, nvm is out.

Second is that docker has a API I can work with (via Dockerode in this case),
makes it easier to use/debug and such.

Third is the easy of isolation. It's basically what docker is about while nvm
is more for just managing versions.

I still use nvm locally, when developing. But when I want to rapidly test in
multiple versions, I use docker with autochecker.

------
geophile
Dumb question, from someone who doesn't work with Node.js: Why are there 32
versions of it?

~~~
tlrobinson
There are actually 303 versions of node (including iojs, according to the Node
Version Manager):

    
    
        $ nvm ls-remote | wc -l
             303
    

Most of those are patch releases. There are 47 minor releases and 6 major
releases:

    
    
        $ nvm ls-remote | grep -oE 'v\d+\.\d+' | uniq | wc -l
              47
        $ nvm ls-remote | grep -oE 'v\d+' | uniq | wc -l
               6
    

Most language implementations probably have similar number of releases.

~~~
coolmitch
yeah -- perhaps there's a lot of negative things to be said about node, but
the fact that they've released multiple versions? really?

[https://clojure.org/community/downloads_older](https://clojure.org/community/downloads_older)
[http://www.scala-lang.org/download/all.html](http://www.scala-
lang.org/download/all.html)
[https://www.python.org/downloads/](https://www.python.org/downloads/)

etc...

~~~
colanderman
I don't think the issue is that they've released multiple versions, it's more
that they've had so many _backward-incompatible_ releases.

Python and OCaml have had _3_ each. Libc has had 6. 32 for something that's
been around less than a decade is a bit excessive.

Are all these 32 actually backwards-incompatible?

~~~
AgentME
There have only been 3 major releases with known backwards-incompatible
changes. The reason you would want to test your code in so many versions of
Node would be to check that you aren't depending on newer features that aren't
in older versions that you want to still support.

Testing across 32 versions is probably very overkill for most people though.

~~~
AgentME
(Oh, I meant to say 3 major releases since 0.10, which I arbitrarily picked as
what seemed to be the earliest version still in any significant use today.)

------
jdjb
Very nice project. I built something similar for PHP:
[https://github.com/vectorface/dunit](https://github.com/vectorface/dunit)

~~~
diggan
Oh, that's very nice. Thanks for sharing this here, always nice to see how
different people implement the same thing basically :)

~~~
tdvorak
I've implemented something similar to be able to run tests with different
MongoDB versions:
[https://github.com/variety/variety/blob/master/test.sh](https://github.com/variety/variety/blob/master/test.sh)

------
caffeinewriter
This is definitely something that's needed, by myself, and many other
developers. Determining module compatibility across versions is always a pain.
Great tool!

~~~
diggan
Thanks a lot for those kind words!

Yeah, it was needed by myself as well, so I had to scratch my own itch for a
while.

------
diggan
Thanks everyone for the feedback and discussions, been very helpful and also
motivating to see that people like/dislike the project in different ways!

As a small update, I've now made autochecker completely language agnostic, so
you can test basically anything you can put in a docker container. There is
some examples on how to use this here:
[https://github.com/VictorBjelkholm/autochecker/tree/master/e...](https://github.com/VictorBjelkholm/autochecker/tree/master/examples)

Again, thank you all for taking the time to give feedback, I'm forever
grateful to the HN community.

------
pedalpete
Have you thought of running this as a service? Would be nice to just through
an NPM module at it and make sure it will run across multiple versions.
Without having to install docker, etc. Not sure I'd pay for it, but maybe
somebody would...

~~~
greglindahl
Other languages have this sort of test service available for open-source
modules. One example is cpantesters.org, which is integrated into Perl's
package repository. Not only is it easy to check what passes in your
environment, it also makes it easy for module authors to fix bugs in
environments that the author has no (other) access to.

~~~
diggan
Funny you should say so, the origin of the idea behind autochecker, is
actually CPAN's and CRAN's way of automatically test packages.

------
full-of-foo
Awesome job! Can't wait to try it.

An aside - over the last few months I began running my containers on two
popular CI services and have had some serious pain. I have a slight feeling
you will regret the words: "works well with CI as well!"

~~~
diggan
Hah, we'll see! Did some initial testing on Circle, Travis and Jenkins but
still have a lot to test left.

Maybe I should be more careful with my words in the future...

------
jgillich
A few months ago, I wrote something similar with the goal to test on multiple
Linux distributions:
[https://github.com/jgillich/chimera](https://github.com/jgillich/chimera)

It however doesn't execute the containers simultaneously, that is really cool!

------
rrobukef
Would you implement binary search? A feature like git-bisect could be usefull
to find breaking changes.

~~~
shiift
I might give it a go. That was one of the first things I thought of!

~~~
diggan
By all means, please do :) Would be very interesting in having this as a
feature.

------
rcarmo
I can't help it - the first thing I thought of was doing a mashup of this and
leftpad as a service :)

------
topherbullock
Finally... a place to make sure my left pad module works cross-version.

------
orliesaurus
Nice one Victor :)

------
raould42
Frankly, I feel like this is an indictment of all software development,
everywhere. When you need to do this kind of a thing, you are doing it wrong.
I do not mean I am doing it any better, I have the same issues e.g. working on
Android, but there are probably some fundamental things we could be doing to
avoid all these horrible symptoms. Things like NixOS come to mind, where
people are actively researching and experimenting with possibly better
alternatives. I pray somehow somebody smart and motivated can find some
fundamental solutions instead of this rube goldberg overkill type answer.

~~~
andreasklinger
Wouldnt the only solution to "see if you code worked with changing apis" to
either

1) never change apis even if you were wrong or you have more opportunities or
2) know everything beforehand and implement the api perfect

~~~
ludamad
also 3) have enough time to implement all the features after having this magic
insight. A lot of people take for granted that successful software makes tough
choices to ship

------
erikpukinskis
This is why I hate ES6. JavaScript used to be a write once, run anywhere
language.

We sacrificed that for syntactic sugar.

~~~
aeharding
Then just write es5. It'll still run everywhere forever.

~~~
erikpukinskis
That's my plan. But it means I can't use other people's modules anymore.

~~~
mschuster91
Why? Put a transpiler inside your build chain.

~~~
erikpukinskis
This almost always leads to broken debugging, which I think is a bad tradeoff.

Most node developers seem to not use a debugger, but I think that's a huge
productivity sacrifice.

