What's the best way to simulate browsing programmatically? I'm aware of Mechanize / Mechanoid, but not of anything else. Is there anything that supports recent JavaScript and DOM?
Mechanize is good and watir for ruby, and then there's Selenium. Be very cautious of selenium and similar because they invalidate a large amount of QA tests (depending on what you're testing). Selenium works in 1 of 3 ways, injecting javascript into the page via a Firefox plugin, injecting javascript via a proxy and having you add the javascript to your pages. Depending on what you're testing, the addition of the javascript is a change to the page (ie, invalidate the testing depending on what you need to verify). Watir is good as it programmatically controls the browser in the same way a user would, sending the events to the items. However, it's difficult to do a large volume of tests in a reasonable time frame. WWW::Mechanize on the other hand allows you to do as many tests at a time as you like, as fast as you can connect to the server. The difference is that with Mechanize, you don't render the page at all, just process the returned HTML (and javascript elements if you choose to fetch them as well).
Many of the "state of the art" tools are really glorified "recorders", using specific wait times rather than being able to determine when the document is actually fully loaded. Additionally, many of them require you to not use the system doing the testing for anything else, as they require the browser to have focus and the mouse to be non-moving. Mechanize and Watir don't have this requirement (and I don't remember about Selenium, it never fit for the tasks I had).
The final fit I have used is Mechanize for core functionality (page loads, load testing, spidering, link checking, non-cache download times, proper javascript detection, proper text returned, etc.), Watir for interactive functionality testing (browser hidden) and Watir for visual testing (browser not hidden, just sit there and stare at it clicking away).
There can easily be others that I haven't dealt with (I haven't had to do web app testing in a year or so), but from my experience, Mechanize and Watir filled the role very well.
Many of the "state of the art" tools are really glorified "recorders", using specific wait times rather than being able to determine when the document is actually fully loaded. Additionally, many of them require you to not use the system doing the testing for anything else, as they require the browser to have focus and the mouse to be non-moving. Mechanize and Watir don't have this requirement (and I don't remember about Selenium, it never fit for the tasks I had).
The final fit I have used is Mechanize for core functionality (page loads, load testing, spidering, link checking, non-cache download times, proper javascript detection, proper text returned, etc.), Watir for interactive functionality testing (browser hidden) and Watir for visual testing (browser not hidden, just sit there and stare at it clicking away).
There can easily be others that I haven't dealt with (I haven't had to do web app testing in a year or so), but from my experience, Mechanize and Watir filled the role very well.