
Ask HN: Web Dev Testing Best Practices - grumps
So I'm making the jump from a 150K employee company to being employee #18 at a web design agency.  Part of my role, will be:
-Defining/Documenting how designs should work (aka setting up Requirements Managment)
-Process and Development definition(SDLC definition, refinement, deployment)
-Setting up and running QA (essentially, from what I know it's tested by the Dev team)<p>What I'd like to know, is how is everyone testing there webapps.  Most things are built with Drupal or Sitecore (.NET).
======
mootothemax
For testing the front-end, take a look at Selenium: <http://seleniumhq.org/>

It's a bit of a pig to get used to initially, but I guarantee the first time
you run a test and find a newly-introduced bug, it'll make your day - and your
team's, for that matter.

~~~
gregdetre
I'll second that.

Furthermore, Sauce Labs offer servers running lots of different OS/browser
combinations that you can run your Selenium tests on.

In other words:

\- we write our Selenium tests in Python (assert that when you click, X
happens)

\- they get run on our continuous integration server every time we push to
GitHub

\- the Selenium tests fire up IE 8 on Windows on a Sauce Labs server and run
themselves

\- the output gets piped back to us as part of our unit testing suite,
emailing us if any of the tests fail

Well, we _almost_ have it working as well as that :) Either way, we've been
really impressed by Sauce Labs so far.

------
TheFuture
I work at a web design agency about that size. Good luck finding any budget
for testing. Clients don't care, and if you budget for it most will go with a
lower bid somewhere else.

As someone else said, it's basically account people clicking around, you fix
stuff, they click more until they feel comfortable presenting it to the
client. Then you find out that some bigwig at the client has a 5yr old
blackberry where the layout doesn't work right, and that becomes your focus of
your debugging for a week.

~~~
mgkimsal
I'm reminded of the time that I built an ordering system, and had to rebuild
part of it before launch because it didn't work on the client's CEO's webtv.
Yes... webtv. Given that he was the only one using it, and he _wasn't going to
be ordering from himself_ , I tried bravely to suggest that we not rebuild the
front end to work with webtv, but it was not to be. :(

------
stefanbutlin
If you'll forgive the self-promotion, you might like to check out my web app,
Testpad (<https://ontestpad.com>), as a lightweight tool for organising manual
testing. It's basically structured checklists rather than formal test case
management, and has been doing well at agencies both big and small. Because
it's lightweight and flexible (think cross between Excel, Notepad and
OmniOutliner) you can start out as simple as you like and iterate the
complexity as and when it's appropriate.

And to answer your question: I am of course using it to test itself! Plus some
selenium for basic automation, but without going overboard as the app is very
javascript-heavy and interactive, which limits how much can be usefully/cost-
effectively achieved with automation.

------
jonstjohn
Strive to test on three levels:

1 - Unit tests, low-level method testing w/ mocked or no external dependencies
(such as db)

2 - Integration tests, test against external dependencies, such as db, apis

3 - System tests, front-end testing that covers all parts of the system using
a tool such as Selenium

Terminology varies and differentiation of levels, as well, but this can serve
as a good guideline.

You'll need to have everything automated and integrated into a build system
where you can reliably kick off a build and see the results.

A note on Selenium: any non-trivial setup is going to require a fair number of
resources, both in time and environments. Also, you will have to dedicate time
to maintaining tests and your UI changes. However, the pay-off is quite large
when you can run and end-to-end set of tests on your application and know that
you didn't break something.

edit: formatting

------
slashdotdash
I would recommend using the Selenium WebDriver browser automation framework
(there are C# .NET bindings). This can be used in combination with a BDD
approach using SpecFlow, which is also for .NET, or Cucumber for Ruby.

Here are two useful resources where you can learn more about these and
successful automation approaches.

* The Secret Ninja Cucumber Scrolls (<http://cuke4ninja.com/toc.html>)

* Ideas for efficient BDD with SpecFlow through examples (<http://ndcoslo.oktaset.com/t-4861>) a very recent presentation, from NDC 2012, where the SpecFlow author goes through a couple of testing approaches.

Learn about the "Page Object" model approach to automated web testing. It's
important for creating reusable, encapsulated tests that can evolve with the
system under test. This is vital for advancing past the initial, small
automated tests to much larger suites whilst still keeping them maintainable.

------
hluska
I tend to believe that QA should focus equally on function and experience.
Consequently, I'd:

1\. Unit test both front and back end code. This thread is full of great
examples for tools that will make that process easier.

2\. Use a tool (I like jmeter) to frequently test how your project performs.
You would be surprised how often small changes result in massive slowdowns,
especially under load.

3\. Bring basic usability testing into the QA fold. Web dev is extraordinarily
client facing - projects live and die according to the whims of clients.
Consequently, if you bring things like simple hallway testing into the
development flow, your whole team will recognize how important it is to make
things elegant and simple to use.

------
benjaminwootton
I like to think of test coverage as a grid.

On the horizontal layers you have unit testing, integration testing,
acceptance testing - which are tests at various levels of abstraction
throughout your code base.

I then like to test the vertical slices tiers of the application - usually
views, controllers, models, database etc. I like to see unit tests,
integration tests, and 'acceptance tests' focusing in on each of those tiers
to build confidence in the coverage of your testing.

Automation is absolutely key, and getting those automated tests into your
continuous build and release process is definitely worthwhile.

------
sophacles
For backend testing, I like to write tests using nose in python, using the
awesome Requests library to drive the web interaction. Because Requests
handles cookies well (for whole sessions), and allows all http methods, and
has great header, file, post params, etc support, it really makes driving
webapps simple. A couple of functions to get a session into appropriate state
(logged in with user or whatnot) and you have easy setup fixtures -- so you
can try things in isolation quite well.

------
wilsoniya
Interesting article about unit testing as it relates to API development:
[http://blog.seatsync.com/2012/06/unit-testing-as-path-of-
lea...](http://blog.seatsync.com/2012/06/unit-testing-as-path-of-least.html)

Unit tests of lower level back-end functionality is always a good idea, but I
find it actually makes API development simpler because it frees you from the
problems and complications inherent in testing via your target client.

Disclosure: I wrote the article.

------
nirvdrum
One of the problems I really wanted to tackle was ensure consistency in web
rendering across browsers and over time. Basically, automating the process of
"does this page look right?" That culminated in my startup (Mogotest) and the
concept of Web Consistency Testing. Neither are right for everyone, but check
them out. Feedback is always appreciated, too, even if it's about how I use
Bootstrap :-)

------
bwh2
Most agencies that size do very little testing. Typically any testing is
manual and done by a mix of developers and account managers. Not saying that
you have to operate that way, but be aware that automated testing is a
relatively foreign concept for most agencies.

I try to write unit tests for internally developed libraries. For some
projects, Selenium tests will be written as well.

------
datr
Drupal has its own version of SimpleTest for testing code:

<http://drupal.org/simpletest/>

Core ships with a lot of tests and most of the larger contributed modules do
too. They're pretty helpful to run frequently to make sure you haven't broken
anything but they won't test your configuration or any of your front end
stuff.

~~~
grumps
I'm more concerned with testing less commonly used modules, and the front end.
I'm newer to the web dev word, but not new to SW development and testing.

------
notjustanymike
Take a look at Adobe Shadow (good writeup here:
[http://www.webmonkey.com/2012/03/adobe-shadow-simplifies-
mob...](http://www.webmonkey.com/2012/03/adobe-shadow-simplifies-mobile-web-
testing/))

It'll let you refresh changes on all your devices at once.

------
thousande
For JavaScript unit testing I have been pleased with Siesta so far
(<http://bryntum.com/products/siesta/>). The standard version can be automated
using Selenium or PhantomJS.

------
bbsabelli
Individuals and interactions over processes and tools. Working software over
comprehensive documentation. Customer collaboration over contract negotiation.
Responding to change over following a plan.

------
recycleme
Learn to use the dev tools in Chrome.

Here's a great start: [http://jtaby.com/2012/04/23/modern-web-development-
part-1.ht...](http://jtaby.com/2012/04/23/modern-web-development-part-1.html)

------
grumps
Just wanted to thank everyone for their input.

So from what I gather, eveyrone is a fan of selenium. I do recall it being
brought up during my interviews.

Does anyone have a favorite resource they like? i.e: eBook, blog etc

------
damoncali
When you type the words "web design agency", it makes me think that the right
answer is "do not test". The projects more often than not are just not big
enough to justify it.

------
jabo
A friend of mine introduced me to CasperJS and I love it. It's built on top of
PhantomJS.

