This is a guest blog post by Carey Fletcher, a Test Manager with QualIT in New Zealand. She has over 10 years of testing experience working in various sectors with her expertise in software test processes and practices. Carey contacted Janet to get feedback on how she had implemented her process to see if she was on the right track. We decided that it would make a great blog post and asked to write about it. This is her story and we think there are many lessons that can be learned by large organizations.
Deploying a Central Test team in scaled agile – A first-hand view.
By Carey Fletcher
My first exposure to agile testing had me “diving in the deep end”, attempting to define a test strategy for a complex insurance system which was transitioning from a waterfall to agile methodology. At the time there was minimal discussion in the testing community on how you would approach system integration/UAT testing within an organisation with a scaled agile approach.
After a discussion with Janet Gregory at a conference (co-author of “Agile Testing”) and armed with the “Agile & QA organisation structure” used by Hartford Group, I decided the most effective approach was to convert the existing waterfall testing team into a centralised QA team.
The Central Test team was a hybrid waterfall and “post agile development” team. At the beginning of the month we were delivered a large release from the external vendor’s sprint teams. This release was functionally tested from an end to end perspective over a 5-week period. During this period the internal agile teams, working at a 2-week sprint iteration, were deploying “Potentially Shippable” releases into the Central Test team for system integration testing (SIT). The 6th week was a dedicated regression, at a system integration/UAT level. Using this approach, we had regular 6-week releases deployed into production, for all components requiring system integration.
The central testers and agile teams worked together to define the scope of the additional “post agile development” testing, including targeted regression, system integration, UAT, non-functional or any additional “…ibility” testing. The central and agile team’s one-page test plans and the “definitions of done” has been met, provided the business owners and production release management with a clear understanding of the quality of each release.
There were of course significant improvements that could be made to increase speed to production, by adopting further agile practises within the Central Test team, but we had at this point proved the testing organisational structure and testing flow in an agile environment worked.
How we changed and adapted the processes:
- Initially we were repeating some of the agile team’s testing in the central environment because the quality of the releases was low but as the quality improved we moved to the agile team smoke testing their own releases and the Central Test team executing the agreed “Post development” testing.
- The business assigned a risk rating factor to all the end to end business processes and scenarios (positive and negative) to assist with defining a risk based approach to the targeted regression required.
- Introduction of automation, initially the smoke tests; API verification; and business process with a high-risk rating.
- Introduced scrum of scrum of scrums, which was a nationwide view of all development and included the Dev Ops managers, production release management and the Central Test manager. The scrum of scrums had a project level view, where as the scum of scrum of scrums had a release management view.
- A Test Environment Manager was appointed, to ensure the central test environment is managed as a “Production like” environment.
This Central Test team approach was successfully applied again, by QualIT, during the system integration and UAT of a Government system which was used by every person in New Zealand. The agile development approach allowed for functional testing; integration testing with adjacent components and an early view of performance testing, but a central team was required to verify the complex data flows across all the components of the system, from a functional and non-functional perspective.
I am often challenged when applying a Central Test team strategy, in a project which has adopted the agile methodology, claiming that I am “old school”, applying waterfall principles which will provide a barrier to delivering working software frequently. This would be true if there are minimal integrations between potential shippable releases but in a complex system with multiple business process integrations, the Central Test team approach works effectively in parallel with the agile development teams, providing that additional testing support, which is not possible within an individual scrum team.
I really like this article as this showed a real-world success story, by being flexible, pragmatic and smart on tailor the best practices from multiple software development process to suite the environment/people rather than try to follow a process blind folded and end up in a failure.
Thanks a lot for sharing the ideas. we do have the similar problem, now we are applying the microservices to develop a product. but still, there are a lot of dependencies between different teams. To mitigate the integration risks, now we are implementing an e2e user journey test. I am also being criticized by many people that this kind of strategy is “old school” and “destroyed the whole microservices idea/advantage”. Unfortunately, we have not found any better “agile” way to ensure the integration works well. But I am glad to see this approach is really working in your organization and your team. 🙂
I think the collaboration between the agile team and the central test team is the key to success. if the communication are not working well between those two part, the post-integration would be really risky. what is your opinion?
Thank you for sharing an informational article. As we also faced the same issue in testing, I found this article good enough. Now I am going to apply this solution.
Great way to adapt agile testing strategy in a complex environment. It seems to me your main challenge was to make sure quality were being built into the project deliverables, so the strategy to create a central quality gate addressed it quite well.
Some questions pop in my mind like what metrics were used and how the teams kept addressing both bugs raised by Central Test Team together with requirements from new releases.
Great post!
…scrum of scrum of scrums had a release management view…
is this supposed to be like that?
This is an experience report on how they adapted their process to meet their needs. I’m not sure that “supposed to” enters into that. Processes are meant to be adapted as you find out what works and doesn’t work for your teams. It worked for them, but might not work for someone else with a different context.
The agile testing strategy is a great way to adapt to complex environments. My understanding is that your main challenge was to ensure quality was built into project deliverables, so the strategy of creating a central quality gate was very effective. Thank you buddy to share this great blog.