It’s no secret that the majority of organizations struggle with implementing continuous testing (CT) strategies. In fact, one of the biggest stumbling blocks for implementing DevOps is the lack of CT.
To help organizations harness the power of CT, earlier this year, we released our book – Continuous Testing for DevOps Professionals – combining CT insights from the best and brightest in the industry. We cover everything from the fundamentals of CT to the future of CT.
As a follow-up to the book, we sat down with Perfecto’s Founder and CTO, Yoram Mizrachi, to discuss his thoughts on the key drivers for CT, the main roadblocks teams need to overcome and what is ahead for the future of testing and software development.
1. How do you define continuous testing?
Continuous testing (CT) is the ability to embed testing as part of the normal DevOps process. At a minimum, CT should happen daily. Non-CT means there are different rhythms for development and testing and this, naturally, creates conflicts, sync errors and could result in a waste of resources and time.
2. What are the top business use cases for continuous testing?
CT is an enabler for DevOps practices. In today’s testing landscape, you can’t really run an Agile/DevOps practice if your testing is not continuous. The practical question in this case is “what are the business use cases for DevOps?” If you have a business case for DevOps in your organization, CT is a must-have and not a nice to have.
3. What is the key driver for CT?
Like any other element in the DevOps cycle, everything should be automated and non-attended. DevOps is built on the practice that the machine and processes are keeping the rhythm. This means that at any moment in time, teams have full visibility into where they are in the process. Day 1 of the sprint is no different then day n-1 of the sprint. The process of automated continuous integration/testing/delivery ensures this is the case.
4. Who benefits the most from CT?
In DevOps practices, testing is fully owned by the feature team. This team is developing, building, testing and releasing continuously. There are teams which assign testing to developers while others are assigning this to testers. Both are valid practices. Eventually, the main benefits go first to the feature team leader so he can make decisions based on facts and avoid surprises. Group leaders, dev leaders and business groups managers should actively promote the use of CT as it aligns with the overall goals for DevOps practices. Eventually, it will give them much better visibility of the development status during the cycle and instead of only at the end of the cycle.
5. What are some roadblocks in the adoption of DevOps and CT into practice?
The biggest roadblock is establishing trust. For decades, automated testing suffered from a lack of trust as part of the process. Getting to a point where red means red and green means green is essential for fully adopting – and embracing – CT.
Another important roadblock to overcome is being able to quickly identify the source of the problem and solve it. This goes back to “always green” and when it’s not, teams should be able to spend minutes triaging the situation and understanding what to fix, where to fix and go back into the normal rhythm.
6. What do you say to organizations who think they can’t engage in continuous testing?
DevOps isn’t for every team – its dependent on skillset, organizational structure and business demands. Think of it this way – if you have not exercised in two years, no one would recommend running a marathon as the first step to get back into shape.
That being said, DevOps practices are becoming the main trend in development, and businesses that do not adopt and practice DevOps will ultimately end up in an undesired place in terms of people, skillset and business goals. Unless organizations have really specific or unique needs, there is no such thing as “can’t engage.” Today, when it comes to CT, it’s a question of “how” and not “if.”
7. Where should organizations who want to get into continuous testing start?
Small. Start small and focus on building a working practice that you can trust. Don’t create hundreds of scripts and dozens of platforms from the get-go. Organizations who are starting out with CT will not see success by taking this approach. Instead, start with 20-25 scripts, about 4-5 platforms nightly and scale slowly from there.
8. What does the future of testing and software development look like in 5 years? 10? 20?
Looking ahead, cloud-based testing labs will become a standard practice, just like Amazon Web Services (AWS), Azure and GCP are for normal IT. The movement in code to higher level building blocks – from native to HTML to react – will also affect testing and we will see more packaged testing infrastructure that supports easier building of testing.
Because of the nature of applications, we will also see artificial intelligence (AI) being used more frequently in testing as teams work to overcome more and more unknowns and unexpected elements during the testing period.
Finally, we will see more supporting analysis frameworks, likely based on AI/machine learning, which will guide developers and testers on what should be fixed first.