Mobile and web projects are becoming more and more agile, with manual testing methods being replaced by automated processes. It’s a sign of the times: unattended automation in DevOps pipelines is needed for successful development projects today.
But which testing tools should you use? Answering this question is critical to building the proper testing strategy and to the continuous testing activities through each phase of the software development lifecycle (SDLC) pipeline.
The Two Main Types of Testing Tools
Before getting into too much detail, let’s take a look at the types of testing tools of the automated testing platform spectrum. These tools can be divided into two major groups:
▪ White-box — Tests written by the developer with access to the code, flows and native objects.
▪ Black-box — Tests written by external test engineers without access to the code and using flows which simulate the user experience.
For mobile test automation, there is a big difference between these tools. White-box testing relies on an IPA/APK package that includes both the app under test and the test suite that is needed to test the app — these packages are executed on the device upon installation.
Black-box test tools are executed remotely and communicate with the application via the network (mostly via HTTP).
White-box tests are executed on the device, which makes the execution faster — up to eight times faster. They have full access to the application code and thus have direct access to objects, making scripts much more stable. Both black- and white-box test tools are needed for the different phases of the SDLC.
Now that we understand the fundamental differences between the tool types, let’s look at the DevOps processes.
Matching Testing Tools to Each Phase of the SDLC
Organizations transition to Agile in order to release more features faster and meet customers’ digital experience expectations. This new velocity concept added complexity each phase of the SDLC. DevOps is the software engineers’ solution to that complexity.
The DevOps pipeline describes the actions from writing the first line of code until it is delivered to and used by customers. This process defines actions, stages, and when to move between stages.
The foundation of an efficient pipeline is automation. Automation can include:
- Building the software.
- Copying, moving, and deploying the app or microservice.
- Automated testing to verify the artifact’s readiness to proceed to the next stage.
The way to achieve this level of efficiency is by employing automated testing at each stage in the pipeline. Such tests have different targets, as described below.
Standard Phases of the SDLC
Developer Local Build — Component Team
Developers write code and then integrate it with other team members’ code. After finishing a feature, and before committing the code, the developer must verify that the component is still stable. This means getting the latest code, merging changes on the local machine, and executing unit tests. This ensures the latest addition did not break the team’s code.
Hourly Build — Component Team
Every hour, component codes are built and all unit tests are executed to verify the code is ready to integrate with other teams’ code.
Hourly Build — Application Build
If all component tests pass, a full application build gets executed every hour. This build contains all the applications’ components. At this point, functional testing verifies that the app is stable and ready to move to staging. Regression tests test it with other apps and services and verify the new functionally hasn’t broken existing features.
Nightly — Integration Test
After verifying that the app is stable, it is moved to a staging environment. Full functional testing has been executed on the app to verify that it works with all other services.
This is the last stage before release to production. This environment contains real production data and tests in this environment are focused on coverage and performance.
After releasing the app, these tests verify the production service is up and running and available to customers.
Keep in mind the other popular tools and frameworks that leverage Selenium/Appium WebDriver configuration and execution. This allows the test developer to focus on application flow and tests. Frameworks include:
- BDD frameworks like Quantum
Such frameworks also enable test developers to add unique test scenarios for specific technologies.
For example, one might use Protractor for specific Angular-based application testing due to the unique capabilities the framework has for object identification support. In a different case, teams could leverage Galen to develop visual testing scenarios for their responsive web application testing.
Including unique scenarios such as these enrich the testing process, and thus, the end product.
Testing Tools For Each Phase of the SDLC
Now that we’ve had a look at the different tests and tools, here is a map of the required test types and testing tools for each stage in the pipeline for responsive web apps:
And here are the required test types and testing tools for each stage in the pipeline for mobile native apps:
Go Forth With the Right Testing Tools
Both black- and white-box tests can help improve overall pipeline quality. Keep in mind that each phase of the SDLC requires a different testing tool that provides relevant feedback at each stage.
As you move to the pre-production stage, the same tests are still relevant and executed. However, the coverage, the environment, and the resources used are different due to the objective. At that stage, the goal is to reduce the risk of escaped defects and improve the customer’s digital experience.
Learn more about your test automation options and check out our article, Selecting the Best Open-Source Test Automation Tool for You.