
Show HN: Nightmare – Simple browser automation - pkrein
http://www.nightmarejs.org/
======
tucosan
Even after substituting the fancy ampersands on your homepage, and requiring
nightmare your example still does not run:

    
    
      var Nightmare = require('nightmare');
      new Nightmare()
        .goto('http://yahoo.com')
        .type('input[title="Search"]', 'github nightmare')
        .click('.searchsubmit')
        .run();
    

yields:

    
    
      node yahoo.js
      phantom stdout: TypeError: 'null' is not an object (evaluating 'element.value = text')
      
      
      phantom stdout:   phantomjs://webpage.evaluate():3
        phantomjs://webpage.evaluate():4
        phantomjs://webpage.evaluate():4
      
      stream.js:94
            throw er; // Unhandled stream error in pipe.
                  ^
      TypeError: undefined is not a function
          at next (/private/tmp/night/node_modules/nightmare/lib/index.js:56:23)
          at f (/private/tmp/night/node_modules/nightmare/node_modules/once/once.js:16:25)
          at Proto.apply (/private/tmp/night/node_modules/nightmare/node_modules/phantom/node_modules/dnode/node_modules/dnode-protocol/index.js:123:13)
          at Proto.handle (/private/tmp/night/node_modules/nightmare/node_modules/phantom/node_modules/dnode/node_modules/dnode-protocol/index.js:99:19)
          at D.dnode.handle (/private/tmp/night/node_modules/nightmare/node_modules/phantom/node_modules/dnode/lib/dnode.js:140:21)
          at D.dnode.write (/private/tmp/night/node_modules/nightmare/node_modules/phantom/node_modules/dnode/lib/dnode.js:128:22)
          at SockJSConnection.ondata (stream.js:51:26)
          at SockJSConnection.EventEmitter.emit (events.js:107:17)
          at Session.didMessage (/private/tmp/night/node_modules/nightmare/node_modules/phantom/node_modules/shoe/node_modules/sockjs/lib/transport.js:220:25)
          at WebSocketReceiver.didMessage (/private/tmp/night/node_modules/nightmare/node_modules/phantom/node_modules/shoe/node_modules/sockjs/lib/trans-websocket.js:102:40)
    

The examples in the githubs pages readme don't run either. I am too lazy to
debug this.

~~~
pkrein
hey sorry about that, just realized i forgot to npm publish v1.1.0 --
published now hope that helps

~~~
zwischenzug
Hey, I'm working on this atm:

[https://github.com/ianmiell/shutit/blob/master/library/night...](https://github.com/ianmiell/shutit/blob/master/library/nightmare/nightmare.py)

which should help matters. Will reply when done.

------
pkrein
Quick example of how we use this at Segment... for each integration
([https://segment.io/integrations](https://segment.io/integrations)) we need
to create a vector logo. This got a bit tedious after the first 50 :) so we
built nightmare + nightmare-swiftly so that our chat bot could "auto-generate"
vector logos: [https://cloudup.com/cEA-dTd3glM](https://cloudup.com/cEA-
dTd3glM)

Hope it's helpful!

~~~
bennyg
Is it doing this by checking colored pixel areas and making SVG shapes for
them? If so, whatever you use should replace Illustrator's Image Trace
functionality, cause that is so bad at creating crisp vector copies of things.

~~~
aarondf
Looks like they are submitting it to Swiftly, and then a human does it.

~~~
bennyg
:( - I was hoping for a cool algorithm to determine that. Haha maybe I'll
write one.

------
ssharp
Could I accomplish multi-page form automation doing something like:

    
    
      .goto(page1)
        // page 1
        .type('selector', 'value')
        .click('selector')
        .wait()
    
        // page 2
        .type('selector', 'value')
        .click('selector')
        .wait()
    

etc. ?

If so, that seems super easy and more straightforward than vanilla Shadow or
even Casper.

~~~
pkrein
that's exactly right

~~~
ssharp
Thanks, this should be absolutely perfect for a project I've been
procrastinating on :)

------
resu_nimda
I've been wanting to programmatically get my transaction details from my bank
for a while. I knew about phantomJS but never got around to trying it out.
When I saw this I decided to give it a shot, and within half an hour I was
able to log into my bank's website (a complicated multi-step process that I
had failed to achieve with just POST requests).

Nice tool, thanks. The debugging could use a bit of work - whenever something
went wrong it just spit out a bunch of "null is not an object" messages from
phantom. Overall it was very easy to use though.

~~~
Lord_Zero
Whats a good way to automate this to run? I see the getting started in the
documentation says "Save it as hello.js and then run it from the command line"
but Iwasnt sure if there are tools like CRON in Linux to automate the script.

~~~
resu_nimda
That is the documentation for phantomjs, which is a stand-alone process.
Nightmare is a package for nodejs (that interacts with a phantomjs process),
so your code runs in a node context, where there are plenty of scheduling
options, e.g. the node-cron package, or just plain setInterval().

------
mknittig
Looks like DalekJS ([http://dalekjs.com/](http://dalekjs.com/))

~~~
tiglionabbit
Or Twill for Python. ([http://twill.idyll.org/python-
api.html](http://twill.idyll.org/python-api.html))

------
wehadfun
I'm using casper now but this seems like it may be even easier.

~~~
meritt
Yeah, there's a ton of "easier to use" libraries atop Phantom. CasperJS is by
far the most recognized and community-backed. Not sure what value all of these
others provide.

------
mherrmann
I am one of the founders of a startup that sells a similar, proprietary
library for Python and Java: [http://heliumhq.com](http://heliumhq.com). Maybe
some of the people reading this will find it interesting.

------
NoMoreNicksLeft
It's a polished website... and I know what javascript is, but they didn't do a
good job of explaining what it is.

Browser automation? Is this a test suite of some sort, something like
selenium? Or is it more like Mechanize?

------
IanDrake
I do like the work they've done with Phantom but the API is horrendous and I
found myself wrapping the parts I used. Nightmare looks more complete and
probably better tested than my wrapper. Well done.

------
ryanackley
Looks like an interesting library. I have no use for it right now but maybe
one day. Definitely can see why this would be useful for some people though.

Their website has some serious style. It looks cool and original. Which raises
a meta-issue that has been bugging me for a while. This is the bar you have to
meet for even an open source project nowadays. It's not enough for you to pour
your blood sweat and tears into something and then give it away for free. You
then have to create slick looking site to market it and get people's
attention.

~~~
annnnd
Not really... Counterexample:
[http://www.haproxy.org/](http://www.haproxy.org/) :)

I think most of the people don't care about the fab wrap, as long as the
product works properly.

------
novaleaf
obligatory self-promotion, for those interested in an easy to use PhantomJs:
[https://PhantomJsCloud.com](https://PhantomJsCloud.com)

~~~
novaleaf
oh, and I'd love to let my users use nightmare in their scrips, but I don't
know an easy way to sandbox the requested phantomjs instance :(

------
thinkvitamin
I use Ubuntu Linux, and I've got PhantomJS installed. Whether I use Nightmare,
Nightmarejs, or Phantomjs on the command line this is what I get:
ReferenceError: Can't find variable: Nightmare

    
    
      nightmare.js:1
      

Should there be more instructions for installing? I believe the brew command
is something for Mac users only...

------
chenglou
Shameless plug: I maintain Node-huxley ([https://github.com/chenglou/node-
huxley](https://github.com/chenglou/node-huxley)), which accomplishes what
this project does, but without the need for writing automation code. Also
compares previous/next screenshots for you automatically.

------
ankit84
How is it different from [http://nightwatchjs.org/](http://nightwatchjs.org/)
?

~~~
JoeAcchino
Nightwatch runs against Selenium, Nightmare uses Phantom.js

With Selenium you have full cross browser support, while Phantom is Webkit
based.

------
thathonkey
I currently only have experience using CasperJS which also wraps-around
PhantomJS. This has different goals (simpler) and therefore a simpler API,
though. The use case at Segment (automatically generating vector logos from a
given graphic) is even cooler though, I think! Thanks for sharing!

------
dutzi_
Nice work!

Can you please provide an example for using `evaluate()` to get the page's
content?

I tried this, but didn't get anything:

    
    
      new Nightmare()
        .goto('http://yahoo.com')
        .evaluate(function (page) { return this.content; }, function (res) {console.log(res);})
        .run();

~~~
pkrein
extracting stuff seems to be a common use case so added an example to the
readme for ya:
[https://github.com/segmentio/nightmare#examples](https://github.com/segmentio/nightmare#examples)

------
andrey-p
And, if you want a very very thin wrapper around Phantom (essentially a
phantom script manager), feel free to have a look at this instead:

[https://github.com/andrey-p/ectoplasm-
js](https://github.com/andrey-p/ectoplasm-js)

Pardon the shameless plug.

------
wearhere
What is the project's background image
([http://www.nightmarejs.org/nightmarejs.org/images/ground.jpg](http://www.nightmarejs.org/nightmarejs.org/images/ground.jpg))?

~~~
AnkhMorporkian
That's a photo that Frank Hurley took on his (nearly doomed) antarctic
adventure with Ernest Shackleton. You can find scads more photos here:
[http://www.kodak.com/US/en/corp/features/endurance/](http://www.kodak.com/US/en/corp/features/endurance/)

~~~
wearhere
Wow, thanks for the info! What an amazing story, and photos.

~~~
AnkhMorporkian
It really is. Hurley is one of my favorite photographers of all time; his
stuff just captured the essence of the time so well!

------
SnowProblem
As someone entirely unfamiliar with both Nightmare and PhantomJS, what is the
typical use case? It isn't obvious to me from the examples or FAQs. Why prefer
these over something like Selenium?

~~~
elwell
I assume because it's more light-weight and hence suited better for certain
needs.

------
dylandempsey
Very nice. FYI best practices are to have a CTA or indicator on your site to
scroll down, so no one leaves thinking the whole site is just an image and
title.

------
brianzelip
Segment usually puts out great tools and crisp sites!

------
nstart
Does anyone know how this compares to zombie js? I'm new to headless browser
testing so apologies if question isn't relevant

------
henrygrew
I'm building a bot using this, any idea of proxies i can use to change the
source ip's and how i can use it with a proxy?

------
tuxone
Fantastic easy tool! There is a way to pass phantomjs command line options
(eg. --ssl-protocol=X ) through nightmarejs?

~~~
tuxone
Found it, line 108 in lib/index.js

    
    
      - phantom.create({ port: port }, function(instance) {
      + phantom.create({ port: port}, "--ssl-protocol=X", function(instance) {

~~~
henballs
Have you tried this? It's not working for me :|

I'm getting about:blank everytime I hit a web address using ssl.

------
brickmort
Extra points for page design!

------
fimdomeio
so... how does one query the DOM? It looks it has been left out of the api.
Does this mean that Nightmare is only able to do brainless automation without
checking browser results?

------
oldboyFX
Interesting background... creepy is good.

------
niix
Very nice!

------
mdaniel
Ordinarily I would file a website bug, but this happens so often I'm
mentioning it here to hopefully stop others from doing it, too.

 _Please_ don't use fancy quotes or the fancy apostrophe for code examples on
your webpage. It looks out-of-place with the rest of the fixed width font and,
most importantly, it stops folks from being able to copy-and-paste your
examples into their terminal or editor.

~~~
pbhjpbhj
Sounds like a browser level or OS level plugin could be useful to check paste
buffers and swap to|from fancy quotes.

------
curiously
the problem with Phantom.js is that compared to an actual browser it still
falls short. I've given up on Phantom.js all together because it is actually
not a true replacement for Selenium running on an actual browser. I found it
far more reliable to have Firefox running on Xvfb for when I'm testing without
worrying about Phantom.js behaving wildly.

~~~
smellf
It's definitely for a different use case. Phantom/Nightmare is for automating
web page interaction, whereas Selenium is for cross-browser testing.

~~~
curiously
You can use Selenium for automation and web page interaction. There is just an
overhead of running Xvfb but it's trivial to setup.

What I mean is that PhantomJS engine still falls short of a real browser.
Advantage of running a real browser with Xvfb is that you won't ever have to
worry about if the page is going to render properly or have it behave the way
you expect it to. The overhead is not of a big deal, in fact it's a tradeoff
for rendering accuracy.

Don't get me wrong headless browser like PhantomJS is headed in the right
direction but if compatibility with website is crucial for your success, it is
not enough and I just don't have a lot of confidence that it will replace a
real browser with Xvfb.

Basically the Javascript libraries are reinventing a lot of the wheels that
have already been done in other languages before the explosion of Node.js.

