
The Siren Call of Automated Browser Testing - jcuga
https://cugablog.wordpress.com/2016/01/21/the-siren-call-of-automated-browser-testing/
======
crdoconnor
>The problem lies in the fact that ABTs are highly coupled with the UI’s
HTML/DOM. ABTs can range from fairly brittle to extremely brittle. The less
brittle ABTs typically use some sort of DOM traversal like XPath or CSS-style
selectors which can tolerate some UI changes like colors and minor layout
changes as long as the changes are via CSS only and don’t dramatically affect
the DOM.

This is the reason I follow this pattern for automated testing - only use
readable selector IDs (for individual elements) and only use readable CSS
selectors (for items in a group):

[https://hitchtest.readthedocs.org/en/latest/faq/why_just_htm...](https://hitchtest.readthedocs.org/en/latest/faq/why_just_html_ids_and_classes.html)

This ends up leading to tests that look a bit like this:

    
    
      - Click: view-button
      - Verify text:
          item: first first_name
          text: Sanders
    

Where such readable ids or classes are missing I either add them myself or ask
the front end to add them.

I've tried most other methods of automated testing too (selenium IDE, page
object), and this one is the most straightforward and sensible IMO.

I would probably use this for BDD + ATDD too although I've yet to work on a
project where that's been possible.

>The time it takes to run the tests is often exacerbated by a lot of explicit
delay statements in the test cases.

This is an antipattern. You shouldn't do it:

[https://hitchtest.readthedocs.org/en/latest/glossary/sleep_o...](https://hitchtest.readthedocs.org/en/latest/glossary/sleep_oriented_testing.html)

Unfortunately, avoiding in most frameworks requires the tester to write
asynchronous multithreaded code (which is beyond most QA developers).

------
lotyrin
I wish more teams could get past the split between roles with QA and DEV, that
the idea was as evangelized as DevOps is.

IF your browser test isn't written in high-level concepts (page objects at the
very least), and if your developer making UI changes isn't getting immediate
feedback to also update the library of test code he's affecting, yes, it'll be
fragile and will suck up too much developer effort. (I'll stop short of
asserting the opinion that the UI developer should be changing the tests'
reflections of his UI components BEFORE making the changes themselves, but I
don't think that position is entirely unreasonable)

It should not be hard to find a balance of engineering practice (reuse,
modularity, etc.) that gets you a degree of maintainability that is economical
(utility provided by the system exceeds that consumed) -- just like any other
kind of code.

Anytime someone messes with an interface, they should have roughly one place
to adjust things for their changes. Dashboard controller and views get
completely rewritten app side? Dashboard page object (maybe) gets rewritten
test side.

Things should be written in the first place for maintainability --
XPaths/selectors chosen carefully, using user-facing labels if possible, WITH
collaboration from the UI side -- no stupid duct-tape shit because you don't
have a good way to get at something (like "the fourth input following the
second instance of the word 'Account'").

Like any kind of code subject to change, you must invest in engineering for
robustness in the face of that change.

No, you can't have a mandate from pointy-hair types to pour unskilled,
unplanned effort into a system or practice because it sounds nice and expect
to break even, but no shit -- that has nothing at all to do with browser
automation.

Depending on the economics of the team, product, even the code-illiterate guys
running the selenium recorder in another timezone and sending in bug reports
any time it complains have their place (for instance, in a year-long release
cycle of mostly stable line of business software where there are few UI
changes because they'd require big change-orders and rewriting all the
documentation) -- you have to design your strategy for your product.

~~~
crdoconnor
>I wish more teams could get past the split between roles with QA and DEV

QA and Product, too. A really good QA is the best person to write user stories
with a customer because they have a natural ability to sniff out edge cases
which they can then walk the customer through.

------
jakozaur
How can we solve that? Perhaps we need a automated browser testing that can be
used by non-developers?

~~~
TheAndruu
Selenium provides this capability with a tool that records where you click on
your website and then it records that as a macro which acts as an automated
test.

However, the issues remain the same as with any other ABT test. An async
element might take a while to load due to latency, and isn't there by the time
your test says to click a button, or browsers were updated and now render your
CSS just differently enough to break the test.

Perhaps part of the challenge is there are so many things that can change and
break the tests outside of the actual code.

------
_greim_
A less sensational title for this would be:

> Thinking realistically about automated browser testing.

But great article none the less.

