The CT recipe: people, process, technology
No matter where you work, you’re probably already using some type of agile method. However, recent research has shown that maturing from agile to DevOps is easier said than done. It turns out that one of the biggest stumbling blocks for implementing DevOps is the lack of Continuous Testing (CT). Without CT, Continuous Integration and Continuous Delivery simply can’t work.
Given that a large majority of organizations are struggling with their CT strategies, we thought it would be a good idea to step back, take a deep breath, and demystify the process of creating successful CT by offering a few high-level best practices to follow.
1) Treat test code as production code
The art of writing good test code requires its own conversation but, in a nutshell: give test code the same amount of love and attention as your production code. Companies that are successful with CT follow this rule. Taking the right approach with test code from the start will make the difference between a high-value test suite and a flaky, inefficient one.
2) Decide which tools you need
“A proper test framework is at the core of test authoring, object management, test management, reporting, and various integrations and APIs for high test productivity.” – Eran Kinsbruner, Continuous Testing for DevOps Professionals
There are lots of good tools out there, some open source, some commercial, but few cover all of these critical areas. Most are tailored to certain development methodologies – it’s essential to choose a toolchain that matches your people and your processes.
3) Maintain stable CI
Continuous testing can’t work if your Continuous Integration practices aren’t working. CI builds should always be green; in order to make sure they are, your teams need to gate what gets into your CI to make sure it’s running smoothly.
4) Define useful metrics
You need to have clear definitions of measurable goals defined as early in the development process as possible. Consider KPIs related to current testing efficiency, application quality, pipeline efficiency, and quality cost measurements. Defining these metrics can be difficult, but you can’t improve what you can’t measure!
5) Use analytics
A commonly-reported weakness of the leading open-source test frameworks is their lack of reporting and analytics tools. You’ve gone to the trouble of defining good KPIs – you need intuitive analytics to see where you stand and where improvements need to be made. Perfecto offers enterprise-grade visual analytics that will integrate with your toolchain, no matter what you’re currently using!
You can get over the CT hump
At the end of the day, Continuous Testing is about mitigating risk. Good user experience means loyal, engaged customers. Bad UX, though… we all know what that means. CT is a mindset focused on ensuring that your customers keep coming back for more.
CT isn’t easy to get right. In fact, most organizations are still struggling with it in 2018 – but don’t despair! There are proven best practices, methodologies, philosophies, and techniques that will guarantee success in your CT endeavors. Following these 5 tips is a start, but if you want to really dig in and understand the nuts and bolts of Continuous Testing, be sure to check out Continuous Testing for DevOps Professionals by Perfecto’s own Eran Kinsbruner. Written by Eran and with contributions from several top pros in the testing world, Continuous Testing for DevOps Professionals is a how-to for understanding and implementing CT in your organization.