The mobile space is highly dynamic, fragmented, and challenging for QA professionals like yourself – especially when it comes to any new major release.
On the one hand, these releases provide the opportunity for rich user experiences and improved engagement. However, these advancements mean that testers have to deal with the consequences — namely supporting multiple OS families, devices, and generations.
With so much changing with each release, a fearsome question arises: How is this complexity managed?
The key to success is this: understanding user flow and applying it throughout your mobile automation testing.
User Flow in Today’s Mobile Testing Landscape
In the fragmented mobile landscape, app functionality varies across devices and operating systems. Depending on different target environments, apps needs to be tested in different ways. What does this mean if you are an automation test engineer and you need to maintain existing test solutions across devices, OS families, and generations? First, consider user flow.
User flow is the series of actions a user takes in order to accomplish something.
It’s critical to testing, as it upholds the functionality of the app from the user’s perspective. And it ensures the most common functions are represented in testing – functions that vary between devices, operating systems, and generations.
User flow may differ drastically depending on the capabilities and supports of the OS and device. For example, in order to login to a device, you may use your fingerprint, face identification, or just manually enter a username and password.
Take for example a Twitter mobile application. How you login, how you post a tweet, and how you bring the app to the foreground would differ between operating systems, devices, and generations. Take a look at some of the differences here:
As you can see, the user experience will differ depending on a multitude of factors. Because of this, tests need to be based off the user journey, which in many cases can be attached to specific devices and generational capabilities.
Mobile test automation needs to address these different capabilities in order to provide complete test coverage and the optimal digital experience for users.
But how is one to do that? Usually such situations cause “branching” within test suites — an unnecessary complication. However, this can be avoided if one dynamically probes the available capabilities.
To do that, one must examine an app’s metadata and property files.
Digging Into Data for Optimized Mobile Automation Testing
After analyzing the user journey, the next step towards mobile automation testing is to understand the app functions and capabilities. App metadata and property files will be infinitely helpful here. The first step is to statically analyze the files.
1. Statically Analyze App Metadata and Property Files
Both Android and iOS have application-wide property files. These define sets of supported app features, required permissions, and other information. This information is located in the AndroidManifest.XML file for Android and in the Info.Plist file for iOS.
These application resources should be statically analyzed during test planning. Knowing what a tested app supports in terms of functionality and user interaction is crucial to the success of the whole test cycle.
A lot can be learned when an iOS IPA (a deployable archive containing iOS apps meant to be run on real devices) or an Android application package (APK) file is unpacked.
Examining the application properties reveals which of its features are available across different devices – critical knowledge for understanding the user journey.
2. Dynamic Analysis of App Metadata and Property Files
Now that static analysis of application capabilities and other metadata can be conducted by using simple text editors and dedicated tooling, how can this be done at runtime?
A library or utility that could be referred to as a dependency by test solutions would facilitate the dynamic analysis of supported application features. It should be able to:
- Open a packaged app.
- Load its application metadata in memory.
- Provide the means to query/introspect application metadata with regard to some interesting capabilities.
Thanks to Perfecto system architect Dror Kedem, there is already an open-source solution for Android OS. It is publicly available on Github. To get started quickly with the tool, you just need to follow the steps below:
- Create a Java project in your Java IDE of preference.
- Add the Java files that you will find in the aforementioned Git repository.
- Modify the “main” method in “AndroidManifestReader.java” so that it points to the correct path of your APK.
Running this file as a Java application will dump into the “Console” view of the IDE everything that can be retrieved from the Android manifest file bundled with the APK.
Thus, one quickly learns that the analyzed application requires billing permissions, external storage write permission, multitouch screen, Internet connection, or other things of that nature. The relevant activities that are used by the app are listed as well.
Now armed with this critical information, you can better understand the app’s functions in the context of the user journey. With this knowledge, you can create more comprehensive test coverage.
Knowing what application features are supported for any given version of a tested mobile app is essential for better mobile automation testing. It can help you avoid unnecessary and complex “branching” of test suites.
The information one can extract from property files and metadata unleashes the potential for better coordination, understanding, and agreement among testing and software development units.
By combining this data with insights from the user flow, one can create a nuanced test strategy that is both effective and thorough. The context of the user journey will only improve test coverage — which can affect your bottom line, and ultimately make or break a digital experience for users.
Appium, Espresso, and XCUITest are the leading iOS and Android test automation frameworks. Get more familiar with them — read more here: