Integration tests simulate a user using your app - tapping on controls, manipulating views, typing in text, etc (see the video in the blog post). As these steps run you have simple tests to make sure the app is behaving as you expect it to.
Unit tests exercise more discrete things like model objects, controller logic, validations. Tons of small tests like "when we add these money values together, is the sum what we expect?".
At Square we use KIF for integration tests and GHUnit for unit tests. We run both on a continuous integration system (CruiseControl.rb) that runs the tests whenever we push new code.
At my current company we have a rather large automated test infrastructure that builds and runs ~50 or so individual libraries and their associated unit test programs on around 20 different configurations for 10 different platforms. Most of these platforms have a way of doing an unattended deployment of our test programs (and associated data) to the device, execute the programs, and retrieve the output.
This isn't working on iOS though because of its dependency to Xcode. We've tried scripting Xcode with AppleScript but this is fragile and routinely breaks.
Has anyone tackled this problem and succeeded?
- clones the repo from git
- builds with xcodebuild
- signs/provisions with xcrun
- creates a manifest file for over-the-air installs
- copies the build to a server where users can install
- saves off the symbol file for our crash server so it can symbolicate crashes
- sends emails
2. Writing the UIAutomation scripts is really tedious (though
it's much better in iOS [redacted]).
3. The UIAutomation tests ran very slowly.
That said, just like UIAutomation, KIF stands on the shoulders of the great accessibility features in iOS. In many ways KIF is like an Objective-C version of UIAutomation.
Are accessibility labels the only way to reference views?
We're still considering the best way to implement gestures. There's some rudimentary support, but it's incomplete right now. Your input is welcome!
If the login failed for some reason, then step (4) would fail because it wouldn't be able to find the refresh button.
You can also add explicit verification steps, such as "wait for the view with the accessibility label 'Refresh'" if you want further validation.
Seriously, unit testing in iOS is not great, and this kind of simulation is exactly what we need. Regular unit tests are great for model-layer stuff and data-processing code, but they will not help with most real-world problems that occur.
"Does my login flow still work?" and "Can I still use every feature of my app successfully?" is the kind of thing you need to know after committing code.