Using the quadrants to plan a pipeline
In Part 1, we showed how the quadrants can guide conversations to plan testing activities that support continuous delivery (CD). What about other tasks to support CD, like building a deployment pipeline? Building pipeline infrastructure involves much more than configuring containers and implementing a continuous integration (CI) tool. It also means creating the infrastructure as code and making sure the team learns the new skills required. A lot of thought and experimentation goes into it. The quadrants model can help teams make sure they are ready to take on these activities at the right time.
Consider each stage of a deployment pipeline. Where would it fit in the quadrants? Static analysis and unit test stages are technology facing tests guiding development with the fastest feedback. What do you need to make that happen?
The team may need to spin up a test environment in the cloud to deploy a release candidate and run performance tests related to the technology facing tests that critique the product. you are there tests that can be triggered automatically? Does the team have a good understanding of what they are?
After deploying to production, monitoring to detect if error thresholds have been exceeded, could be considered a business facing test that critiques the product. Perhaps the need to deploy to the idle “blue/green” production environment and run a smoke test suite or perform some exploratory testing, would fit in that same quadrant. As with the more traditional use of the quadrants, there are no hard and fast rules. The conversation is what matters.
One team’s experience
A few years back, Lisa’s team decided to move from deploying to production every two weeks to deploying twice a week. The team already had thousands of automated tests at all levels and had already implemented a cloud-compatible CI tool. They had switched hosting of the site to the cloud and were automating the deployment process.
After starting the twice-weekly deploys, the team realized that insufficient exploratory testing was resulting in bad bugs getting out to production. Some team members set up dashboards with monitoring tools, but most people on the team didn’t know how to use them. They often didn’t even realize if the deployment pipeline had a failure, and when they did, they weren’t sure how to diagnose and fix it. They scrambled around looking for better ways to refresh test databases for the automated testing stages to run against. Automated regression test coverage turned out to be poor and many regression failures occurred.
In hindsight, if the team had taken time to have conversations using the quadrants, they may have thought to plan some important activities before switching to frequent deployments. The example below shows some things they might have planned. Defining service level objectives (SLOs) and service level indicators (SLIs) early in the planning, would have enabled confirming these in the staging environment. They might have even come up with the idea to spin up a cloud test environment for each new build, with a unique name reflecting the change to be tested.
Potential benefits
As we’ve learned from Jerry Weinberg and others, most software problems are really people problems, and many of those are caused by poor communication. A thinking tool like the quadrants gives teams a framework to build a common language and a shared understanding of what they need to deliver. It helps to foster a whole team approach when considering testing activities and working to build quality attributes into our product.
The quadrants are a great way to facilitate a conversation among people from different specialties: product people, designers, operations specialists, testers, business analysts, developers. Bringing a diverse group of people together is powerful. The different perspectives and encourages lateral thinking, and maybe offset each other’s unconscious biases. It helps prevent blind spots and shorten feedback loops.
Once teams make risks visible, they have a chance to mitigate them before they escalate into problems, and that is important. Using a tool like the quadrants, teams can improve their deployment pipelines, plan ways to gain confidence in delivering small changes more frequently, and improve their runbooks to help team members dealing with production issues.
Thank you Janet and Lisa for sharing your team’s experience and benefits. It’s actually helpful.
Very good blog