Here is btw a demo playground to try it out: https://chromeless.netlify.com/
Let me know if you have any questions :)
I spent a while looking at this a few months ago to do a similar project (run chrome in headless mode on AWS Lambda) but couldn't get the binary to work.
If you're interested in how serverless-chrome works, I wrote an article on how to get headless chrome running on AWS Lambda from scratch here: https://firstname.lastname@example.org/running-headless-chrome-on-...
similar idea. chrominator use promises instead of a fluent api. it also follows the selenium w3c spec where possible. it does cool stuff with evaluate and evaluateAsync where it resolves the remote object to something usable.
to be fair there are a few other projects i know about that wrap chrome remote debugger with a high level api:
* autogcd 
* ghostjs 
To echo another comment on this thread, headless chrome seems well-positioned to shake up the automated browser testing market. The price—especially with the AWS Lambda free tier—is very, very compelling for a number of projects.
Please feel more than welcome to take a stab at it yourself and create a PR for this feature! :)
Btw, does the .viewport() option not work in the demo? I'm seeing a `TypeError: Failed to fetch` when I set one.
- "Do pretty much everything you've used PhantomJS, NightmareJS or Selenium for before".
The main features of those tools plus their ability to handle a large range of edge cases are built up over the years in production use and do not seem to be already in Chromeless. Also, Lambda costs can be a significant point of consideration for professional test automation with large volume.
Nevertheless, there's no turning back as flood gates have been opened and many developers are noticing Chromeless. I believe, with enough dedication from Chromeless maintainers, they may be able to channel the attention and contributions to shape Chromeless to be the main challenger to existing test automation approaches. That will really be a blessing to the open-source community!
The only catch I believe, is it may be easier for those existing tools to be made working in Lambda or implement a similar form of parallelism while still having their mature API, than for Chromeless to catch up to the state of maturity of those tools. But as they say, growth solves almost every problem, so issues like these may be ironed out through collaborative efforts from contributors/maintainers.
I totally agree with you! It took years for these tools to mature and so it will be the case for Chromeless. There are probably a range of edge-cases that yet have to be solved but like you said, I'm very optimistic that together with our great community, we'll be able to handle all of these cases.
The big incentive for us to create Chromeless instead of using Nightmare or similar (which I've done for years) was the fact that you can now use headless Chrome (which provides a way more stable foundation) + the ability to execute the code on AWS Lambda which solves the parallelisation question. I hope this makes sense to you :)
Yes perfectly, I believe for any one very serious about test automation or browser automation in general, they will make their own tool and bring it to market if there isn't already one that meets their needs. :)
I'm doing something similar, and this concern was one motivation for running in our datacentre vs EC2.
Does anyone have concrete info on rates of bots blocked from AWS IPs?
What are sites' motivations for blocking AWS IPs? I bet there are some reasons I would agree with even though the somewhat crude method of blocking ip range would have some unintended consequences (e.g. blocking people running a personal VPN).
I block AWS. So many crawlers up to so much nonsense!
I don't block by IP, but by hostname.
$ua = @$_SERVER['HTTP_USER_AGENT'];
if (stripos($rh,$block)!==false &&
$message="Blocked Host:Amazon Web Services";
And then all the off-brand scraping companies use amazonaws.com.
Cliqzbot, VidibleScraper/1.0, CheckMarkNetwork, CCBot/2.0 (http://commoncrawl.org/faq/), linkdexbot/2.2; +http://www.linkdex.com/bots/
That last one is a "SEO platform".
You can then use the vm/container as a function to match AWS lambda.
Is it the that the api is more-user friendly or selenium w3c complaint?
Genuinely curious, don't know much about this project.
There's work being done to fix this, but it's still in progress, last I checked. Until then, resizing windows, taking screenshots and a handful of other things simply don't work.
EDIT - above is assuming downloading a file by simulating a click event to perform the download. there may be other workarounds by script injection etc to use XMLHttpRequest() for downloading a resource directly.
The promise of fast execution time in parallel is tempting with Chromeless. Thanks for sharing.
Ultimately it was the combination of using headless Chrome and the ability to execute code in parallel on Lambda, which made us invest in Chromeless.
The error report given by this are some of the best.
We've built something similar internally and will shortly migrate it to Chromeless. We basically use it to pre-render our websites and docs: https://github.com/graphcool/prep
The first two examples seem to return 404, for me:
Seems pretty awesome.
My use case is to take screenshots of various pages - the docs don't mention the default viewport dimensions, btw.
(since I've ran into the same trap a couple of weeks ago and ended up with USD 650 of unanticipated charges): the free Lambda tier includes
* 1M requests
* 400k GB-seconds
--> the GBsec can be a serious bottleneck. Imagine you're running each Lambda instance with 512MB RAM and each instance takes 2 mins to complete your test. This means that one Lambda instance is ~61.5 GBsec, meaning you can execute ~6,500 of these instances per month to remain in the free tier.
Depending on how extensive your tests are/how often you run them, you might run out of free GBsec well before you'd run out of the requests quota.
However assuming a 8 hour work day you then get ~27 instances per hour. Each test takes two minutes to run, so for a single user testing, assuming a code - test - code - test routine, you'd be able to do that nearly continuously, for 8 hours a day, every day of the month (no weekends or days off). Seems safe to assume that wouldn't occur.
The Chromeless Proxy service uses the @serverless-chrome/lambda package as is. Same build.
Any plan to support other languages beside JS?
Some API documentation says:
pdf() - Not implemented yet
Not implemented yet
How about you deprecate the API for now but reveal the purpose please.