The three-way disconnect
When an application vendor develops its Android or iOS app, usually the mobile app development team would mark in the property files what features it uses and supports (i.e. GPS, TouchID, Doze, etc.) and what permissions this app requires, such as HTTP connection, GPS and File system.
For the app developer, once these property files are developed, his role in defining the app capabilities is done — which now pushes the test responsibility to the testing teams.
The problem that now arises is, how do these testing teams associate the right test cases based on the app capabilities that are only supported by certain mobile devices?
The screenshot below and to the right is taken from a Google Android Developer guide. It’s an example of a line that would usually be in an application Manifest.MF file within the App.APK file. What it’s showing here is that this specific app would require the use of the phone’s camera – this will trigger the user to allow camera access in the app’s first launch post-installation.
While that’s a common case, what would a tester do in a case where the requirement is for TouchID, ForceTouch, Android Doze or Multi-Window (coming in next Android N release)?
These features would complicate the test scenarios and test automation because only specific devices and/or OS versions would support these new OS capabilities. Without automatically planning and knowing which test should be executed against which device/OS combination, there is simply no way to manage these tests in today’s mobile app landscape.
As a result, many organization will lean on manual testing as a way to overcome the test delta for newer features that are unique to device/OS combinations. But when this gap grows, test cycles slow down and if not addressed quickly, the balance between manual and automation tests weighs toward manual as more enhancements are introduced in the app.
In the image above, you can see how a “write post” functionality would be tested on various iOS devices – when you test for iPhone 6S/6S Plus, which supports ForceTouch functionality, these devices would have to be tested differently than older iOS devices and iPads. In addition to the traditional GUI test within the Facebook app, tests for writing new posts through ForceTouch would need to be added just for these devices. The same test scenarios would be required for any other menu item in the app.
Break the app-device-test disconnect
The main point here is to continuously make sure that the test automation suite supports all app capabilities, and is able to be executed against the right mobile platforms based on all the supported capabilities.
So how can you overcome this challenge?