
Announcing Testling - Automated Cross-Browser JavaScript Testing - cleverjake
http://www.catonmat.net/blog/announcing-testling/
======
asolove
Let's make an n-dimensional map of the available javascript testing
environments. The dimensions are something like:

\- headless v. javascript driving browser v. full browser-level

\- js-only v. test-specific html page v. load full app html

\- local v. cloud

And the current contenders go something like:

\- Selenium: browser, full app, local

\- Sauce: browser, full app, cloud

\- FuncUnit: js-browser, full app in iframe, local

\- qUnit: js-browser, sparse html page, local

\- JS Test Driver: js-browser, js-only, local

\- ad infinitum...

The result is that you need (at least!) three different tools to do three
different but required tasks:

\- unit tests that run in a quick-feedback, autotest style on the command line

\- module-level tests that require a small bit of DOM access and cross-browser
testing

\- integration tests that require loading the full app and triggering events
on it.

It would be better if we could come up with a convenient, simple abstraction
that allowed doing all three, and sharing code for mocks across the three, and
configuring in what environment and what machine they each run.

Ideally, this could be done with AMD and wire.js. Just write a test in
javascript, declare its dependencies, and they get injected.

\- Unit tests depend on just one or two js files. They get loaded and then the
tests run.

\- Functional tests depend on "dom," provided either by a browser or a
zombie.js-style headless dom, and a variety of js and html snippets. They all
get loaded and run.

\- Integration tests depend on "app", provided by actually running against one
of your servers. It spins up, the tests run, maybe there is an API for telling
the test server to reset itself to a known state between tests.

~~~
substack
We pretty much have what you're describing already, except we use node.js-
style require()s with browserify instead of AMD. You can write really simple
unit tests and just execute them with a curl one-liner or you can write more
integration-style tests for an existing app with t.createWindow() and
t.submitForm().

~~~
asolove
Interesting. But how do I run tests with t.createWindow locally? Do I need a
publicly-accessible IP? We are currently looking at Sauce because we can write
tests against Selenium and run them locally or in the cloud, plus they make it
easy to tunnel in and run against things in our internal network.

~~~
substack
Right now, you do need a public IP for t.createWindow(). We have ssh tunnels
in browserling already but they don't work with testling just yet. Stay tuned.

I think one of the biggest benefits of testling right now is that you don't
have to install anything outside of curl to start using it.

~~~
asolove
Fantastic, I look forward to it. I started this post mostly frustrated by the
idea of someone creating "yet another" js testing tool, but now I'm
legitimately interested in seeing where this goes. Thanks for the fast and
helpful feedback.

------
benblair
Looks very useful as a github post-commit hook for catching browser-specific
JavaScript errors.

Is this also meant to be used for testing usability across browsers? Like
browserling but automated? If so, some specific examples on the docs page
would be very helpful.

~~~
substack
Yep you can use it both ways. To drive an existing page around you can use
t.createWindow() and t.submitForm(), which give you a new window object and a
jquery handle bound to that window object. <http://testling.com/docs/#an-
example-test>

------
dshankar
Woot! Congrats on the launch. Time to add Testling to the StackVM site, eh? :D

~~~
pkrumins
Thank you! Yep, got to add it there.

Edit: added! <http://stackvm.com>

------
rokhayakebe
Great products, Poor designs. The last part takes a lot away from the first,
making it almost look as if this is a side project. Just saying.

~~~
jesusabdullah
I've heard people say this about browserling, and I can kinda see the point,
but I thought testling's landing page was kinda pretty. Maybe I'm just weird.

~~~
rokhayakebe
You are weird. Just kidding. It just kind of hurts me a little bit to see a
great product come short in design.

Edit: Best HN username goes to you.

~~~
jesusabdullah
> Best HN username goes to you.

<3

> It just kind of hurts me a little bit to see a great product come short in
> design.

I do have to wonder what the design would look like were a "pro designer" to
get their hands on it. I think SubStizzy did a pretty admirable job, but man
design is _hard_. I know I'm pretty terrible at it anyway.

------
rafaelferreira
Can I run jasmine on testling?

~~~
substack
The plan is to support other test styles like jasmine and qunit through npm
modules: <http://testling.com/docs/#using-npm>

------
gregwebs
How do I run tests locally without using this service? If I could run these
same tests most of the time with qt-webkit (phantom.js) and then run on
testling before deploying I would use this.

------
hugs
For other NodeJS JavaScript testing options, there's also Jellyfish
(<http://jelly.io/>)

------
hugs
Congrats on launching! Welcome to the arena. :-)

~~~
substack
Hey thanks! The business of making the web a little less terrible and a little
more reliable is a great space to be in.

------
acak
Nice work.

What I would like to see is automated testing on different mobile browsers (on
real devices).

Anyone have pointers for such tools? Thanks.

------
jackfoxy
Great news guys! I'm going to make sure my team finds out about Testling.

------
mcs
Node.JS!

~~~
pkrumins
You bet!

