
Chrome 59 has cross-platform headless support - vvoyer
https://www.chromestatus.com/features/5678767817097216
======
bluepnume
This is fantastic. I'm using a combination of Chrome and PhantomJS for karma
testing right now, for [https://github.com/paypal/paypal-
checkout](https://github.com/paypal/paypal-checkout) and
[https://github.com/krakenjs/xcomponent](https://github.com/krakenjs/xcomponent).
There are hundreds of tests opening up hundreds of iframes and popup windows,
and sending a lot of cross-window messages, and that ends up being really
memory hungry.

Chrome deals pretty well with garbage collection, so long as I'm careful to
de-reference closed windows properly¹, and only uses a maximum of 150mbs.
PhantomJS eats up almost 6GB of memory before it's done, which makes it almost
unusable on machines with less memory or CI boxes. Travis is a no-go.

I'm hoping running Chrome in headless mode should give a nice speedup for our
tests.

\-----

¹ Turns out even a closed popup window or iframe keeps a huge amount of memory
hanging around. Who knew.

~~~
paulirish
We (Chrome) have reached out to PhantomJS to inquire if they're interested in
collaborating: [https://groups.google.com/d/msg/phantomjs-
dev/S-mEBwuSgKQ/tU...](https://groups.google.com/d/msg/phantomjs-
dev/S-mEBwuSgKQ/tU06ZOZcFgAJ)

The DevTools Protocol is the primary API for headless Chrome, but we are
excited for higher-level abstractions like PhantomJS & NightmareJS's API to
manipulate the browser as well. Plenty of details to work out, but hopefully
sometime this year you'll get a drop-in solution for some of your testing to
upgrade from Phantom's older QTWebKit to the latest Chromium.

~~~
2T1Qka0rEiPr
A few hours later:
[https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...](https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE)

~~~
yamaneko
After his announcement, Vitaly answered to the team:
[https://groups.google.com/d/msg/phantomjs-
dev/S-mEBwuSgKQ/PQ...](https://groups.google.com/d/msg/phantomjs-
dev/S-mEBwuSgKQ/PQdE1A5FAwAJ)

> >Is there interest on your side in adopting Chromium as a runtime? There's
> some existing documentation [2] around the API and embedding, but
> admittedly, this would be some work.

> We are interested. But I am afraid not in the current state. Currently,
> PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt
> Chrome as a new runtime.

> But I think we could implement PhantomJS as a completely new (with the same
> API) project that will use Chrome - Phantomium!

~~~
Naracion
Also see
[https://github.com/Vitallium/phantomium](https://github.com/Vitallium/phantomium)

------
nreece
I've been testing Chrome headless extensively for the past few months, and
while it's a good step, but it's not stable for high-volume or even diverse
set of webpages.

Memory usage is pretty high, lot of heavy webpages result in crashes/hangs,
there are many inconsistencies between features available in full version and
headless, their debugging protocol has different APIs that work on
headless/non-headless in Linux or Windows, and so on.

Of the bugs I've submitted, some have been fixed in the upcoming M59, so other
critical ones may take longer due to their backlog. I suppose for now (maybe
until M61-62), Chrome full with xvfb or even PhantomJS are better options.
When you realize that Chrome is about the same size (by LoC) as the Linux
kernel [1], you can't help but wish for a leaner & faster headless browser.

There seems to be some work going on building Firefox pure headless as well.
Great overall, as long as all the browsers try to follow the RemoteDebug
initiative [2].

[1] [https://josephg.com/blog/electron-is-flash-for-the-
desktop/](https://josephg.com/blog/electron-is-flash-for-the-desktop/)

[2] [http://remotedebug.org](http://remotedebug.org)

~~~
jotto
I've been successfully using Chrome headless in a 500MB Docker container for
dumping the DOM for
[https://www.prerender.cloud/](https://www.prerender.cloud/) for months
(rendering a large variety of sites without restarts for weeks at a time)

Run it with:

    
    
      --js-flags="--max_old_space_size=500"
    

to force the VM to keep it GC'd below 500

Chrome v55 was a 30% memory savings, before that I used 1GB containers.

It's not perfect, but I am definitely pushing high volume (multiple tabs,
concurrent activity) and I am not having any significant stability issues and
I am pushing diverse sets of webpages.

~~~
vvoyer
What the best strategy that worked for you on high volumes, multiple tabs or
multiple docker instances? I am wondering if multiple tabs is as efficient as
multiple windows/instances.

Thanks.

~~~
heipei
I'm using more than one Chrome process so I can kill the processes every so
often (e.g. after timeout or when they get stuck). Inside each Chrome instance
I use 16 tabs, there might be a number of factors at play:

\- Are you worried about same-origin pollution if you run multiple tabs from
the same origin in the same process? If so -> Extra process \- Do you have to
take screenshots? You can only take screenshots of the tab that's in the
foreground, so you have to activate it first to take the screenshot. This
might fail if you have lots of tabs which roughly trigger at once.

You can see what I've built at [https://urlscan.io](https://urlscan.io) btw.

------
callumprentice
[https://bitbucket.org/lindenlab/dullahan](https://bitbucket.org/lindenlab/dullahan)

I've been working on a fully open source Windows/macOS library (via Chromium
Embedded Framework) that allows you to render pages to memory (and then of
course to bitmaps, textures etc.) as well as inject synthesized
mouse/keyboard/JavaScript events. It currently uses (what amounts to) Chrome
57.

Looks like this might make my project obsolete.

~~~
keville
Thank you anyway for your efforts; I hope you interpret this as a validation
of your having correctly identified a sorely-needed tool.

~~~
callumprentice
Thanks keville - I've poured my heart into this on nights and weekends after
I'm done with my day job for a long time now but what you say is so true -
hopefully we'll get access to a more robust solution compared to my modest
hacks.

------
MichaelApproved
A lot of phantom js talk here makes me want to recommend
[http://ghostinspector.com](http://ghostinspector.com)

It's a phantom js (and other headless browser) web service. Using the site,
you can quickly create different tests, scheduled tests, chained tests, keep
screen shots, create videos of multi step tests, and have historical
information of it all.

Can't say enough good things about the site.

Edit: also there's a great chrome extension that will record your mouse clicks
and keyboard commands to make creating a test that much simpler.

~~~
guiambros
+1 to GhostInspector; I used them at a previous company a few years ago, and
it was very useful.

They were just starting, but service was rather reliable, and their tech
support was excellent (maybe because we were early customers). We used to run
a bunch of automated tests for monitoring and compliance, archiving hourly
screenshots over different builds for later comparison.

------
vvoyer
Also checkout [https://github.com/cyrus-and/chrome-remote-
interface](https://github.com/cyrus-and/chrome-remote-interface) for an easy
way to fully control those headless instances

~~~
kenshaw
If you're using Go,
[https://github.com/knq/chromedp](https://github.com/knq/chromedp)

It's also worth noting that the 57+ series has a nice embedded viewer you can
use to view the actual viewport via devtools.

~~~
chrissnell
WOW. That's fantastic. As I noted in a comment above, I wrote a tool called
crabby that uses Selenium and Chrome headless to do automated page testing and
to report results back to metrics engines like Graphite, Datadog, Prometheus,
and Riemann. The biggest problem I have is the unreliability of chromedriver
and the extreme resource consumption of Chrome + Selenium. It's really too
much for your average public cloud instance if you want to test any more than,
say, one or two pages a minute.

Do you know if chromedp can access any of the timing measurements?

~~~
kenshaw
Yes, you can access everything via the underlying APIs. chromedp is a
relatively new project (only about 4 months old), so there isn't much yet in
the way of high level timing / profiling, but we hope to add that to the code
base when we have some bandwidth to do so.

------
skibz
The feature I was most interested in when they announced this last year was
virtual time. The Developer Resources link has it listed
([https://chromium.googlesource.com/chromium/src/+/lkgr/headle...](https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md))
but it's a broken link, unfortunately.

Mostly, I'd like to know how the control of the virtual time system would be
exposed. Would it be through the C++ API, or could it be made available
through the debugging protocol?

------
ctphipps
Any way of scripting this to automate button clicks etc? I use PhantomJS for
this now but found it to be incredibly unstable for complex pages.

~~~
sandstrom
There is an excellent library called nightmare (based on electron). If you
need print-screens you can use xvfb

or wait for this issue:
[https://github.com/segmentio/nightmare/issues/224](https://github.com/segmentio/nightmare/issues/224)

~~~
symtos
it should be noted that nightmare isn't safe for untrusted websites:
[https://github.com/segmentio/nightmare/issues/1060](https://github.com/segmentio/nightmare/issues/1060)

------
fake-name
Shamelessly bumping my project to produce a nice python API for the
Chromium/Chrome-remote-debugger-protocol: [https://github.com/fake-
name/ChromeController](https://github.com/fake-name/ChromeController)

I'm trying to replace PhantomJS in my infrastructure with chromium. Not having
to build my own chromium will be a very nice thing.

------
vmasto
I've been trying to test audio and video with headless browsers (namely
PhantomJS) but have experienced extreme difficulty, I wonder if headless
Chrome is able to support/supports already HTMLAudioElement or
HTMLVideoElement or any media interface that would make, for example, testing
YouTube or SoundCloud embeds easier.

------
swah
Related: I want to take screenshots of a few news websites for a little fake
news project of mine, and most approaches return something completely
different than what I'm seeing when I open Chrome.

Limited height would be better/ok (something like the first 3000 pixels).

Low volume / can be slow (30 seconds would be ok).

Those news websites many times have infinite scrolling.

I've tried:

\- phantomJS (rendering sucked, tried every technique I could find to wait for
JS to load)

\- wkhtmltopdf (almost ok, generates a huge 30M image with all the height, no
antialiasing it seems)

\-
[https://github.com/gen2brain/url2img](https://github.com/gen2brain/url2img)
(this was the best so far, uses Qt bindings but not the latest version)

\- actually run a headless browser in DigitalOcean with xvfb-run and take a
screenshot: I failed at this

What I didn't tried was Selenium, because it seemed even harder.

How would you guys do it?

~~~
stangls
I use xvfb for selenium testing. It is really easy to set up and can take
screenshots or videos while automatically browsing websites.

~~~
swah
Thanks - I tried Selenium (on the desktop) with geckodriver now and it
rendered well. The only thing is that long screenshots didn't work but there
is probably a workaround for that.

------
yarp
Any chance for webgl here? Would be nice for automatic screenshots and webgl
tests.

~~~
jevinskie
It is being discussed.

[https://docs.google.com/document/d/1VTcYz4q_x0f1O5IVrvRX4u1D...](https://docs.google.com/document/d/1VTcYz4q_x0f1O5IVrvRX4u1DVd_K34IVUl1VULLTCWw/edit?disco=AAAAAxbGHos)

------
Bahamut
Oh my goodness I have been waiting for this day for a while - we ran into
PhantomJS problems with keyboard/mouse eventing and the HTMLVideoElement for
testing, this sounds like it should be the cure for our woes of having to hack
around PhantomJS's deficiencies.

------
iAm25626
Nice!! Would creating WebRTC data channel be possible?

server side SCTP to client(p2p over SCTP/data channel) would be cool.

~~~
stcredzero
It's already possible. I'm using webrtc and node-electron to connect a golang
server for my MMO project. I have a farm of 4 nodejs processes running under
tmux acting as proxies for unreliable communications.

[https://www.emergencevector.com](https://www.emergencevector.com)

~~~
unmole
Which WebRTC library are you using on the golang server? Last time I looked, I
couldn't find anything stable.

~~~
stcredzero
I am using electron-webrtc. You need xvfb installed. I was able to get this
running under node, but only running the processes under tmux. You will also
need to install some random shared libraries Chromium needs, but which node-
electron doesn't install, but these are obvious from error messages.

Then I'm using simple-peer on top of that. There's also a library for UDP
communications from the node process to the golang process.

------
arrowgunz
What does this mean for Electron and other apps that depend on Electron?

~~~
JepZ
I am looking forward to cli tools powered by electron. Imagine the beauty of a
grep writting in js, executed by a chrome binary. :D

~~~
ryeguy
Imagine the terrible performance of grep written in js, executed by a chrome
binary.

------
ericb
Would this run as a chrome driver for Selenium? What is needed to make this
work with Selenium?

~~~
onion2k
It should be possible, but as a browser rather than a driver so you'd still
need Chromedriver to glue Chrome and Selenium together. In Karma with karma-
chrome-launcher[1] you can pass options to the browser using the flags option,
eg;

customLaunchers: { chrome_headless: { base: 'Chrome', flags: ['\--headless'] }
}

I've not tested that yet but I can't really see a reason why it wouldn't work.

[1] [https://github.com/karma-runner/karma-chrome-
launcher](https://github.com/karma-runner/karma-chrome-launcher)

------
vvoyer
Almost one year old but there was a talk on headless chrome at the Blink
conference (BlinkOn 6):

Video: [https://youtu.be/GivjumRiZ8c](https://youtu.be/GivjumRiZ8c)

Slides:
[https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcx...](https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcxzMQNEE7PcuQrGu83No3l0lw)

More links: Headless Chrome architecture:
[https://docs.google.com/document/d/11zIkKkLBocofGgoTeeyibB2T...](https://docs.google.com/document/d/11zIkKkLBocofGgoTeeyibB2TZ_k7nR78v7kNelCatUE/)

Mailing list:
[https://groups.google.com/a/chromium.org/forum/#!forum/headl...](https://groups.google.com/a/chromium.org/forum/#!forum/headless-
dev)

All of those links are on
[https://chromium.googlesource.com/chromium/src.git/+/master/...](https://chromium.googlesource.com/chromium/src.git/+/master/headless/README.md)

------
mstade
Fantastic news, not a minute too soon! Can't wait to get rid of PhantomJS. Now
if only this was a standard feature of all browsers...

------
est
Is it possible to install Chromium on a server without X environment? Last
time I checked it requires a shit ton of dependencies.

------
livoras
PhantomJS has plenty unsolved issues(up to 1.7k+), a replacement instead of
combination might be a better choice.

------
tianlins
How fast is headless vs. normal? According to

[https://developers.google.com/web/fundamentals/performance/c...](https://developers.google.com/web/fundamentals/performance/critical-
rendering-path/render-tree-construction)

the chrome browser spends a decent amount of time on other steps such as
parsing HTML. I wonder how much time could be saved by not rendering pages
into pixels.

------
retube
This page doesn't load for me (IE behind corp firewall). How does one
drive/automate a headless browser? What kind of API is there?

~~~
tonyblundell
Docs and a Youtube presentation...

[https://chromium.googlesource.com/chromium/src/+/lkgr/headle...](https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md)

[https://www.youtube.com/watch?v=n6biclFh0i0&feature=youtu.be](https://www.youtube.com/watch?v=n6biclFh0i0&feature=youtu.be)

------
dkastner
I put together an example of how to run chromium with --headless driven by
cucumber/capybara: [https://dkastner.github.io/2017/04/21/headless-
chrome.html](https://dkastner.github.io/2017/04/21/headless-chrome.html)

------
brendandahl
For those interested, Firefox is also going to support a headless mode. The
current nightly supports headless SlimerJS on Linux and more platforms will
come soon.

[https://bugzilla.mozilla.org/show_bug.cgi?id=1338004](https://bugzilla.mozilla.org/show_bug.cgi?id=1338004)

------
hackcasual
Looks like the launch bug is private?
[https://bugs.chromium.org/p/chromium/issues/detail?id=705916](https://bugs.chromium.org/p/chromium/issues/detail?id=705916)

------
smackfu
Have people found many issues that come up in Chrome but aren't found in
PhantomJS? We used to use a headless browser but switched to PhantomJS and
haven't had any real issues.

(We should probably run under the real IE but jut haven't been bothered.)

~~~
adzicg
Phantom does not support more recent JS syntax/tweaks. We have an app that is
aimed at more recent browsers only so can use latest ES6 features, and had to
move from Phantom to in-browser tests (the alternative would be to use babel
for transpilation, but then we wouldn't be testing the code that is actually
released to users)

------
laurencei
Can anyone confirm - would this work with a Flash/SWF application? i.e. could
I use the headless mode to interact with the Flash Application to run some
commands and retrieve the output?

I tried googling around but didnt find much to say either way...

------
du_bing
It seems that Chromium 59 still can not be installed on Raspberry Pi, or
anyone has done it?

It will be great to use this headless Chromium on Raspberry Pi to execute some
routine web browser jobs.

Does it support the extensions installed on Chromium? Curious.

------
amingilani
Does this mean I no longer need to use phantomjs for my tests?

------
wslh
How fast is the debugging mode? I tried the first debugging protocol when
Chrome added it and it was very difficult to use. I assume this time is
different?

------
armitron
Doesn't seem to work on OSX. Connecting to debug port from a different chrome
brings up an empty page.

Running Version 59.0.3069.0 (Official Build) canary (64-bit)

~~~
vvoyer
Works for me when using [https://github.com/cyrus-and/chrome-remote-
interface](https://github.com/cyrus-and/chrome-remote-interface)

~~~
kawsper
Looking at [https://developer.chrome.com/devtools/docs/debugging-
clients](https://developer.chrome.com/devtools/docs/debugging-clients), it
seems we are missing a Ruby client. Last time I tried it gave me some
headaches trying to talk to the websockets, but hopefully someone smarter than
me can pick it up.

------
_pdp_
This is great if the headless mode supports the web extension API because it
means that we can run our security tools almost as command line tools.

------
hbakhtiyor
i use when they announced headless mode on linux, and built generating
thumbnails from captured screenshots of websites and uncovering the
technologies used on websites

and the api is available for free,
[https://github.com/letsvalidate/api](https://github.com/letsvalidate/api)

------
0xFFC
Can somebody explain to me what is this good for?

Thank you.

~~~
vacri
Headless browsers are frequently used in buildserver testing of web apps.

------
stheakanath
Is this confirmed to work with Flash? I know selenium did not support Flash so
it caused some dev issues.

------
MR4D
Given the rough comments on the Electron story earlier, this should be welcome
by all.

------
zigomir
Not sure if this can enable SSR (server side rendering) for any client side
lib?

------
unixhero
What is the use case for headless Chrome?

~~~
ntaylor
> Headless mode allows running Chromium in a headless/server environment.
> Expected use cases include loading web pages, extracting metadata (e.g., the
> DOM) and generating bitmaps from page contents -- using all the modern web
> platform features provided by Chromium and Blink.

Practically speaking, software developers will use headless Chrome to automate
testing of product functionality. Today, developers use systems like Selenium
or PhantomJS to accomplish this feat, but it's a painful process to maintain
these headless browser execution engines. Adding headless support into Chrome
means that developers can count on the presentation of their application on a
given version of the Blink engine run within Chrome.

------
softwarelimits
is this is chromium, too?

------
masterleep
Please let this be capable of generating PDFs from HTML from the command line.

~~~
zwerdlds
FWIW, you can do that now, using phantomjs (which is chrome)
[http://phantomjs.org/screen-capture.html](http://phantomjs.org/screen-
capture.html)

~~~
chickenfries
Also with [https://wkhtmltopdf.org/](https://wkhtmltopdf.org/) and
[http://pandoc.org/](http://pandoc.org/)

~~~
erichurkman
And, if you need much higher fidelity and control of HTML/CSS -> PDF, there's
the fantastic Prince library, [http://princexml.com/](http://princexml.com/)
(nonfree)

(I've been using Prince for over a decade, rendering everything from
prescription labels, packing slips, receipts, resumes, books, and more. It's
great.)

~~~
jamespaden
That's also an API service, [http://docraptor.com](http://docraptor.com), with
a different pricing model that uses the Prince library for PDF rendering.

