1. Use PhantomJS: It easy to script, but it lags over real browser by few years and development is stagnant. Many real website doesn't work in PhantomJS.
2. Use Chrome/Firefox xvfb. You have real browser, but scripting is hard. E.g. even hello world examples like make a thumbnail of a website takes a lot of time to get right.
According to some rumors, headless Chrome existed before even Chrome was released to the public. Google use it to do web scraping. However, though headless browsers are great for developers, they are also great for spammers and ad fraud. So the main developer likely has conflict of interests whether to invest resources into making headless Chrome public.
I'm using a Chrome plugin for that purpose, which complicates the whole setup even further.
And I'm not quite sure (without having it thought through) how I would do something like get the URL of the last permanent redirect and correlate it with the original request. I'm sure it's possible somehow, but it doesn't seem exactly straightforward.
Well, some people also ask why world ever need Ruby with Rails or Python with Django? You can write any website in Enterprise Java since late 1990s. Same Docker vs. Virtual Machines or Atom vs. Vim/Emacs. Even if capability is there, there is room for improvement. Especially if you can make it easier to use and make developers more productive by an order of magnitude.
> IMO it is hard to setup and resource consuming
What issues have you had? It's always been very easy to setup in my experience.
The only two 'issues' I've had are correct management of the Selenium server within a task running context and odd interactions with some browsers when they are driven by selenium (e.g. safari considering `window.open()` to always be a popup from selenium triggered actions).
Btw, can you point me to some tutorial that teach you in 15 minutes from zero how to setup something that can change any url into a thumbnail screen shoot. You can do that with phantomjs:
brew install phantomjs
Maybe a selenium pro can do it quickly, but for me:
1. How to install selenium? It took me few attempts until I realized that Docker containers are the best way.
2. Install Docker on MacOS and start container.
3. Find some toolkit that talks to selenium. There are tons of libraries, but just some have reasonable docs and are well maintained.
4. Configure this toolkit so it can talk to Selenium.
5. Beware if your container die. Also figuring out that website is loaded is also non-trivial in Selenium and default are worse than in PhantomJS.
To get things started simply with JS just follow instructions here: http://webdriver.io/guide.html
The only major diversions from that are to use Mocha or Jasmine for your test framework (instructions on the same site) and to automate starting / stopping the selenium server for which there are grunt/gulp plugins.
headless: Implement screenshot capturing
(based on patch from skyostil@, also sets default window
size to 800x600 to enable basic snapshot support)
With the --screenshot option, headless shell will save a
PNG screenshot of the loaded page.
However sometimes the page still hasn't finished drawing and so I do a couple of requestAnimationFrame's after the DOMCotentLoaded to ensure that everything is drawn to the screen buffer.
xvfb-run --server-args='-screen 0 1024x768x24 -ac' chromium-browser --user-data-dir <url>
One (minor?) benefit over Phantom is having working file uploads, though it would be awesome to have that in Phantom too.
I created this after numerous issues with PhantomJS compatibility and debuggability; Testcontainers instead uses the real browsers, and also offers automatic video recording of test sessions and VNC access for debugging.
Headless chrome support sounds like a good step forward, but if visibility into what's going on is limited then I feel there's going to be some way to go. Perhaps chrome remote debugging support?
1. A library which headless applications can link to to.
2. A sample application which demonstrates the use of headless APIs.