Playwright is awesome, it’s great to see more tooling evolving around it. I evaluated Playwright vs Cypress and Playwright won by a lot. The WebKit support is a big plus. But mostly, Cypress was way too opinionated / prescriptive for my taste, with some deal-breaking trade offs (which, to its maintainers’ credit, they openly document) [0].
Playwright feels much more like you’re simply driving a browser because you are — it gets out of your way like any good tool should, so you can debug your tests instead of your browser automation toolkit. The trade-off of Playwright is that meant, at least early on, that it wasn’t “batteries included” and the ecosystem around it is only just now starting to mature. This is a great time to start using Playwright IMO.
One gripe with Playwright is that you lose the ability to easily deploy your tests to services like Browserstack where you can do true multi-device testing. For that you need to use Selenium, or (recently?) Cypress. Because of the nature of Playwright only targeting specific browser engines, it seems unlikely that Browserstack would ever fully support it.
But there is probably a place for both, with Playwright closer to the bottom of the pyramid and Selenium / Browserstack toward the top for only the most critical e2e acceptance tests. You could also write your tests in such a way that they could mostly run on Selenium with just a few changes to how you orchestrate them, although then you might end up chasing the “lowest common denominator” of what’s available in either paradigm.
As an aside, I’d love to hear the story of how the Puppeteer team left Google for Microsoft while taking their toys with them. I wonder if there is some drama there? :)
Most of this is invisible to end-users of selenium, as it's just a hammer they pick up; the struggles of the maintainers is not (nor should it be) something they have to worry about.
Nevertheless, they probably could get some hay mentioning that their API implements a significant amount more functionality versus every other cross-browser testing framework.
That said, they really don't have to, the work really stands by itself.
It definitely manifests in ways I can see. I struggled for years as an end user of selenium with backwards compatibility issues, bugs and weird browser quirks.
It's pretty apparent that it requires an awful lot of maintenance to keep up with the browsers, so if this really does reduce the workload on them then that ought be noticeable on my end with fewer bugs, quirks and backwards compatibility issues. This is more exciting to me than all of the other features (I'm a bit dubious of the ability to detect if a page has really 'loaded', for instance).
That is an excellent write-up, and assuages the main concern I was still having with Playwright (Playwright itself addressing the main concern I had with Puppeteer, which was its focus on a single browser). Thanks for sharing!
Oh yes! Really excited to see this. I am using ms playwright for e2e tests for https://sqwok.im and have been very pleased with it. I've evangelized it whenever webdriver/selenium comes up in conversation as I think it's a huge improvement - clean api, multiple browser support, just "works".
The one thing that was missing was a purpose-made test runner. I'm using mocha currently, and while it works fine, there isn't a built-in way to e.g. run the test against multiple browsers w/o ugly for loops/before statements (that I'm aware of). It looks like this defaults to running against all browsers so you don't even have to think about it. The screenshot assertions also look nice!
This is built on top of a very interesting project from Microsoft called Folio[0] that I’ve used and while still very beta is shaping up to solve all of the issues I’ve had with jest while keeping most of the things I like about it.
Can anybody tell me about playwright vs puppeteer? The API looks very similar. At first glance it seems to be written by some of the same people. The major difference appears to be supporting different engines.
It looks like puppeteer may now support FF, but not WebKit/Safari.
It just seems like there is probably a story there.
supposedly some of the key contributors are the same people who also built puppeteer and the objective seems to be taking it a step further to support ALL rendering engines [0]
>We are the same team that originally built Puppeteer at Google but has since then moved on. Puppeteer proved that there is a lot of interest in the new generation of ever-green, capable, and reliable automation drivers. With Playwright, we'd like to take it one step further and offer the same functionality for all the popular rendering engines. We'd like to see Playwright vendor-neutral and shared governed.
Cypress appears to have more features dedicated specifically to making testing easier, such as a user interface and various debugging additions and integrations into various other tools.
Playwright is just an API for controlling the browser, and appears to be far more powerful in that regard.
If cypress works for you that’s great, playwright doesn’t have all of the same tools but it does excel at cross browser testing. It runs safari, Firefox, and chrome.
We’re working on plugging in playwright into a cloud service for visual testing @superadmin.so, running multiple browsers in Linux can be pretty hairy to setup.
Been using it at the Fortune 500 and some other projects I work on and folio fixtures (the test runner under playwright-test) is really nice can pass POM, db connection, JWT token getter directly into your test as test function parameters. Test look like jest. Just wish Webstorm/pycharm would get built in integration
The project I work on switched from Puppeteer to Playwright in order to increase our browser coverage for visual output tests about a year ago and we've been very happy with it.
For my last internship, I had to build an e2e testing suit. After studing the various options already on the market, I ended up choosing Playwright and I didn't regret it.
Puppeteer does a few more things, but only works for chrome.
Both of them are essentially wrapping devtools protocols. AFAIK, much of the playwright approach was guided by trying to Hew as closely to the puppeteer approach, but cross-platform. Both the APIs are very similar.
Does playwright work out of the box in CI systems (github actions) or we have to explicitly install the browser binaries? I am planning to use playwright-java (https://playwright.dev/java/docs/intro)
Playwright feels much more like you’re simply driving a browser because you are — it gets out of your way like any good tool should, so you can debug your tests instead of your browser automation toolkit. The trade-off of Playwright is that meant, at least early on, that it wasn’t “batteries included” and the ecosystem around it is only just now starting to mature. This is a great time to start using Playwright IMO.
One gripe with Playwright is that you lose the ability to easily deploy your tests to services like Browserstack where you can do true multi-device testing. For that you need to use Selenium, or (recently?) Cypress. Because of the nature of Playwright only targeting specific browser engines, it seems unlikely that Browserstack would ever fully support it.
But there is probably a place for both, with Playwright closer to the bottom of the pyramid and Selenium / Browserstack toward the top for only the most critical e2e acceptance tests. You could also write your tests in such a way that they could mostly run on Selenium with just a few changes to how you orchestrate them, although then you might end up chasing the “lowest common denominator” of what’s available in either paradigm.
As an aside, I’d love to hear the story of how the Puppeteer team left Google for Microsoft while taking their toys with them. I wonder if there is some drama there? :)
[0] https://docs.cypress.io/guides/references/trade-offs#Permane...