“Plan your work for today and every day, then work your plan.”
~ Margaret Thatcher
When planning for a continuous testing strategy, leadership should have a comprehensive plan for all teams and individuals. The more you tailor each of these aspects to fit your organization, the more likely you are to succeed in your continuous testing efforts.
1. Number of projects and their type (mobile, web, responsive web, progressive web, etc.)
Do you have parallel development teams? A team for native, web/mobile? A team to maintain test code? WIth the never ending number of device and browser permutations, there’s a lot of code to maintain just to keep your current features and user experience functioning properly across all platforms.
2. Team size – dev, test, ops, source code management, etc
Are you fully Agile? Using CI? Are your teams remote? How are you handling source code management, commits, and builds? How about test results data?
3. Team skill sets – development languages, testing framework familiarity
You’ve got one (or many) development team(s) but who’s proficient in what – and which testing frameworks will be the easiest for you to deploy? Are you a Ruby shop? Should you be looking at Watir? Or can you use Selenium and Quantum? There are so many choices available – mapping these choices to your team/tech stack is critical.
4. Software development lifecycle methods – ATDD, BDD, etc.
What methodology does your organization employ? BDD takes a user-centric approach whereas ATDD is more focused on the developer. Can you utilize less technical staff members to accomplish testing initiatives? BDD can make that easier. If you work mainly with developers, ATDD might be the way to go.
5. Technology availability – which tool stack is currently in use and what is missing?
How does your application code relate to your testing code? Can your team build off the same language as your application? What would be the shortcomings?
6. Market trends and analytics in order to support new features, user stories, etc.
Keeping on top of competition in the marketplace, making sure your features are maintaining your user base, planning for future enhancements will keep your product map
7. Lab sizing and coverage requirements
We see some customers running 6 testing cradles to give them some basic test coverage and others running 80+ cradles to get to 90% automation & more complete coverage across browsers and native apps. Consider where your comfort zone is when it comes to test coverage risk — can you test 80% use cases? Can you afford test only 50% coverage?
8. Metrics for success which are achievable, meaningful, and clearly understood
Start simple and set realistic targets. Measure, monitor, and fly-by-wire, making appropriate adjustments to test cases, test frequency, and coverage as needed.