
Zombie.js: insanely fast, full-stack, headless testing - tswicegood
http://labnotes.org/2010/12/30/zombie-js-insanely-fast-full-stack-headless-testing/?utm_source=javascriptweekly&utm_medium=email
======
croby
First, great work on this implementation. I can't imagine all the work that
must have gone into this.

To me, this seems to only solve a corner case in the main problem of JS
testing - testing the DOM without a browser. The first question that comes to
mind is, is a DOM even useful in a headless browser when it comes to JS? For
now, grant that we're only testing presence and manipulation of DOM elements
and no CSS manipulations or accuracy -- which I'd argue is a decent case for
testing. Now, testing presence and manipulation of DOM elements in the
abstract isn't very useful, as different browsers might treat DOM
manipulations differently. Using a headless browser that no one actually uses
doesn't solve this problem.

I'd see a case for this in running something like QUnit[1] test that just test
JS completeness, but again, implementations differ so testing in one
environment that no one actually is using isn't good enough.

The only thing that I've come up with for a solution for actually testing
JavaScript in a robust, real-world way is using QUnit and Selenium[2] to
define the tests, with Testswarm[3] actually running the tests. This setup
allows unit tests to be run against JS by itself (QUnir), mock interactions to
take place to ensure bindings work properly (Selenium), and have these
executed against real browsers in a distributed manner (Testswarm).

I concede that this isn't an ideal solution, and I wish there were more people
working on solutions like Zombie.js that would combine all of these in one so
that JS could finally be apart of our continuous integration testing. The only
(sic) thing preventing this solution is that it needs to have headless,
identical implementations of all the browser's JS engines and DOM parsers in
order to be valid.

[1] <http://docs.jquery.com/Qunit> [2] <http://seleniumhq.org/> [3]
<https://github.com/jeresig/testswarm/wiki>

~~~
davnola
There needn't be One Test Suite To Rule Them All.

I like having something similar to zombie.js during dev, then Selenium during
CI triggered on git push, perhaps with Testswarm scheduled to run daily.

~~~
eclark
Is test swarm even still up? I can't ever seem to get the website to load in a
reasonable amount of time.

~~~
croby
See the 'Setting Up Your Own Swarm' section on
<https://github.com/jeresig/testswarm/wiki>

------
hammerdr
Reliable headless DOM implementations are one of major obstacles for JS as a
first class language. Kudos and thanks to everyone that is working to solve
this!

------
wildmXranat
Does this load external styles, or javascript files? What about the images and
does it evaluate any script tags?

I have done a few albeit scattered attempts at headless or semi-headless
html/js unit tests. What I found out to be a good middle ground is running is
this:

Run selenium server and firefox on a remote server. Trigger tests via a task
queue, which at the moment is Gearman, have firefox draw to a virtual frame
buffer.

What that does for me is cover pretty much every case that a real browser
does, and completeness is key in my point of view. I would like to see how
Zombie compares on the completeness side and then speed.

------
gojomo
Any chance this could run in Rhino or does the node.js dependency demand V8?

Feature for feature, how does this compare to HTMLUnit?

------
pierrefar
As I understand it, the JS DOM implementation does not have the quirks of the
different browsers. So theoretically, you could write a test that passes and
works in browser X but does not work in browser Y.

My understanding is that jQuery spends most of its time dealing with these
quirks.

------
arnorhs
I like the cleanliness of not needing a browser.

But it seems like you'll have to duplicate a lot of functionality and as soon
as there are a couple of things you can't test against or that don't produce
the exact same result then this becomes unreliable.

Am I wrong?

~~~
reid
Simulated-browser tests are more like smoke tests. It's an approximation of
the environment you're running tests in.

Zombie and other simulated-browser systems can be useful because they're
faster and convenient, but especially because they're faster. If you're not
able to test quickly, like the article says, you lose the flow.

Nevertheless, you'll be burned if you don't also test in the browsers you're
supporting.

davnola, posted above, has the right idea: use Selenium with your CI system in
addition to Zombie.

Another idea: make real-browser testing for development faster. I'm working on
a tool that does just that, if you're curious:

<https://github.com/yui/yeti>
<http://reid.github.com/decks/2010/yuiconf/yeti.html>

Like it's peers, my project won't work for everyone, yet. But it's awesome to
see so many people caring about testing web apps in dev _quickly_. Hooray! :)

------
bokchoi
How does this compare to env.js?

<http://www.envjs.com/>

------
EGreg
does this thing actually compute the rendered coordinates properly, as a
browser would? Can I load jquery and call offset() ?

~~~
a-priori
I would guess no, since it includes only a mock of the DOM. But really, are
element locations something you want to test?

~~~
dools
Definitely! But it would then also require a CSS renderer. Automated testing
of something like Decal CMS (decalcms.com) where we provide javascript which
manipulates the DOM but allows the designer to plug whatever markup/CSS they
like into the system means that automated testing is incredibly difficult and
does rely on actually checking for the position (and other computed styles) of
an element after we've run certain pieces of code.

