
Wallaby.js – An intelligent test runner that continuously runs your tests - flipchart
http://wallabyjs.com/
======
TheAceOfHearts
Seemed interesting until I got to the part where it won't be free, and there's
no pricing or licensing information. Why isn't that on top?

Once more information is available I'll probably be interested.

~~~
wallabyjs
It's free at the moment, so feel free to try it. We will announce pricing and
licensing in advance, so there'll be enough time to decide whether or not to
keep using the tool.

~~~
dagw
Sorry, but I've been there done that. "Here, just try it for free", they say.
"What harm can it do?", they say. "Licensing costs will be basically nothing",
they say. "Trust us", they say.

So I try, and I like and I get used to it and then they start with "Oh you
wanted to use that one really useful feature, That will be $5000 extra. Oh,
you're running on Solaris, that will be $10000 extra. Oh you have more than 2
CPUs, that will be $20000 extra. Oh you want to use it Thursdays as well, that
will be $30000 extra".

So now I never ever try software until I've seen a complete price list.

~~~
wallabyjs
Fair enough.

Wallaby.js will be less expensive than some similar commercial solutions for
other languages.

In any case, feel free to sign up and to receive the pricing info as soon as
it's available.

~~~
jimmcslim
So judging by other comments on this article the price ceiling is set at
$160USD :-)

~~~
dagw
He said " less expensive than some similar" software, not all similar
software. So for any given price point all they need is someone somewhere
selling something similar for more.

------
z3t4
Makes me feel bad for not writing tests.

There's very little done for code analyze in JS, I can't even find an editor
that gives me function parameter hints And show me a list of functions. I have
even considered making my own editor.

Coding without such tools force me to write code that is easy to manage and
review though.

~~~
flipchart
Both Visual Studio (from Microsoft) and WebStorm (from JetBrains) give you JS
intellisense. Granted, JS intellesense is never going to give you as much
information as a statically typed language, but Typescript helps the editor as
does comment type-hinting in VS (using something along the lines of JSDoc)

~~~
ygra
You can annotate types within JSDoc, which can help somewhat. But granted,
compared to statically typed languages it's still a clusterfuck. Currently
writing documentation comment translation for a C#→Java/JS compiler and I'm
injecting as much type-system metadata as I can, but IDE and editors still
disagree to what extent they're actually using that.

------
kristiandupont
I've used NCrunch when writing C# code and it makes life tremendously better.
I've been longing for a JavaScript equivalent ever since, looks like it might
finally be here! Hooray!!

------
lancefisher
This looks cool. I'll gladly pay for great tools so they can be built and
improved. Can't wait to try it. I'd love to see Sublime Text support.

------
jfroma
> It runs your tests as you type

I write a lot of tests and I've tried this a lot of times (since it can be
done with any test framework plus a file watcher on any language) but it
doesn't help on my workflow.

The integration with the code editor looks nice as long as it has a way to
"jump to the first broken test", and show all the details of the error. I
remember when I used to work on c# the test runner of resharper integrated and
it was great. My current workflow is, I run the tests in the console and if
some test is broken, I search the name of the test across the project to
inspect.

Parallel test execution in javascript hasn't worked for me, most of the time I
depend on some "state" call it DOM (browser), or call it a test database
(node). The fact that I can assume that the runner run one test at the time
makes my tests and the before/after code simpler.

~~~
wallabyjs
Wallaby.js is very different from a file watcher. File watcher would run all
of your JavaScript tests and it can be very time consuming. Wallaby uses
sophisticated dependency analysis and lots of other tricks to only run tests
affected by your code changes, no more no less, so even in large codebases
with lots of tests the feedback is delivered very fast.

Yep, wallaby can jump to failing test(s), error sources, show error details
inline and much more.

Parallel execution in wallaby works fine with browser "state". Imagine running
a half of your tests in one browser tab and another one in a second tab -
that's what wallaby does but in 6 or 8 or N processes. As long as your tests
set up context for themselves and don't depend on each other (and they
shouldn't) - it all works.

------
dantiberian
NCrunch, a .NET equivalent of this start at $160 USD, I'd be expecting this to
land in the same ballpark to be able to be developed sustainably.

------
BinaryIdiot
Okay this looks pretty interesting. I'll give it a go. Right now my primary
editor is Sublime 3 (occasionally atom.io but it's kinda slow) so support for
at least Sublime would be awesome. I've been meaning to try IntelliJ so maybe
this will give me that excuse.

Thanks for sharing!

------
jdlshore
This looks well-done. One thing that wasn't clear to me was whether Wallaby
supports simultaneous cross-browser testing a la Karma. Ideally, I'd want it
to just work with Karma. Wallaby folks in this thread, can you clarify this
for me?

~~~
wallabyjs
[https://news.ycombinator.com/item?id=9230987](https://news.ycombinator.com/item?id=9230987)

------
deitcher
TextMate or SublimeText support anywhere in the future?

~~~
wallabyjs
It is in our plans -
[http://wallabyjs.com/#plans](http://wallabyjs.com/#plans)

------
prayerslayer
If this wasn't a commercial product I'd ask why it's a separate thing instead
of a PR to karma.

~~~
wallabyjs
These projects' focus is a bit different.

Karma was built to run all of your browsers tests in as many different
browsers as possible.

Wallaby.js focuses on seamless integration with many code editors and on
delivering instant feedback from your (both browser and node.js/io.js) tests.

~~~
jdlshore
So how does Wallaby run your front-end tests, if not in a real browser?

Even if it does run in a real browser, cross-browser issues are common enough
in my work that my TDD loop always includes multiple browsers. I wouldn't be
willing to give that up for what Wallaby offers.

Part of the reason I'm interested in something like Wallaby is that running in
multiple browsers _is_ slow--but if I could run just the right subset, it
would be fast enough for my needs. (And Karma still runs about 200 tests /
second on my mid-2002 MacBook Pro, across 10 browsers, including IE VMs, so
it's not like Karma is _that_ slow.)

~~~
wallabyjs
Wallaby.js supports PhantomJs. It also supports node.js/io.js for node tests,
as well as for front-end tests with something like jsdom.

Karma is not slow, but some of our tests may be. Wallaby just allows not to
run them if we are not editing anything that affects them.

Cross browser issues are always a pain, I know what you mean. However, I have
realised that I can write the bulk of my code without worrying too much about
them.

When I'm writing the bulk of logic, especially with TDD, I found that it's
more efficient to run tests as fast as possible in the fastest engine and care
about running across all browsers much less frequently. Especially if you run
tests as you type.

Later on, closer to the end of the iteration, when preparing to commit my
changes, I surely run all of the tests in all browsers/smartphones/tablets
that I support. Typically it just works, especially when your codebase is
mature, you already know the majority of cross-browser issue sources and have
something in place to avoid them. Even if it doesn't work, the issues are
normally easy to solve. Overall, less context switching, more focussed work.

Anyway, it's just my experience (you can read more about the motivation behind
wallaby.js in my blog post -
[http://dm.gl/2015/01/30/wallaby/](http://dm.gl/2015/01/30/wallaby/)) and we
are planning to support more browser runners, or even just karma as a runner
for wallaby, in future. Please feel free to vote
([https://github.com/wallabyjs/public/issues/29](https://github.com/wallabyjs/public/issues/29))
or create new feature requests.

~~~
jdlshore
That's a reasonable perspective. For me, my thorniest issues have come from
browser incompatibilities, so it's more important to catch and fix issues at
the moment they're introduced rather than later. This isn't necessarily
testing all browsers; just the ones that I know are likely to trigger
differences. For example, I might run IE 8 + IE 11 + Firefox + Chrome + Mobile
Safari.

Fast tests (< 1-5s) have a lot of value to me (although "test when I press
save" is as good or even better than "test as I type," as far as I'm
concerned), so Wallaby is something I'm interested in. I'm more likely to do
manual test sub-setting than give up my cross-browser tests, though.

One potential customer's perspective; you can take it or leave it. :-) I'm not
necessarily a good example of the mainstream. I've voted on the issue.

------
jkot
I have test suite which takes 3 weeks to finish. My test pretty much runs
'continuously' :-)

~~~
richthegeek
Can I ask what is the point of having tests that take that long to run? Being
notified 3 weeks after I made a change that I broke something is pretty
useless to me...

And what language/framework or other reason could possibly contribute to that
long a test suite?

~~~
jkot
Release acceptance test for database engine. Terabytes of data, stress tests.
I have smaller sets which runs in 10 minutes and overnight.

I do not work on SQLite, but here is something similar:
[http://www.sqlite.org/testing.html](http://www.sqlite.org/testing.html)

------
woah
Will you be supporting TAP (Test Anything Protocol)?

~~~
wallabyjs
We will at some point. Right now working on module loaders support
(browserify, webpack), then will be adding more testing frameworks support.
TypeScript/CoffeeScript next. Feel free to add feature requests/vote -
[https://github.com/wallabyjs/public/issues](https://github.com/wallabyjs/public/issues)

------
sspross
interesting sound choice in the video...

~~~
wallabyjs
Always loved to put some epic music in my demos :)

