The test manager should provide some test expertise to the test group. This expertise comes in a variety of forms:
Ability to discuss, mentor, and/or train current employees on general testing techniques.
Ability to look forward to the products in development and planned for development, to foresee what new expertise and training is needed by the group.
Ability to hire people based on current and future testing needs.
Note that this does not mean the test manager has to provide specific low-level tests for the product. A test lead's job is developing tests, not the test manager. As a test manager, you provide leverage to all the testers and to corporate management. You need to define for yourself the difference between mentoring and doing other people's work.
Wednesday, May 6, 2009
Develop Test Strategies
The ship criterias should have a significant impact on the test strategy for the product. For example, if the product is an operating system that has to run on multiple hardware platforms, ship criteria that emphasize the number of platforms the product has to run on, and de-emphasize the specific features of the operating system will push your strategy to one of hardware combination testing, not depth-first testing across many features on one particular platform. The test manager does not necessarily develop the entire strategy for testing, but should have input to the strategy, and participate in reviewing the strategy with the test group.
Define and Verify Product Ship Criteria
As the test manager, we have an opportunity to negotiate the criteria with marketing and the development groups. You want to verify criteria with customers or representatives of the customer. The development group's job is to figure out how to achieve what the company wants to achieve. Using the customer requirements to figure out what to do provides the developers a complete view of the product and how it works. Once the product is defined in terms of what and how, the testing effort can verify how well the product meets the real requirements.
It's important for testers to prioritize their testing so that the product ship criteria can be met. Since very few projects have enough time to do everything, getting the testers enough information for what and when to test is a very important role for the test manager.
Corporate managers need to understand the product ship criteria enough to be able to judge whether the product is ready to ship at a given date. I don't believe the test group should be either the holder of product approval or rejection-that role belongs to corporate management. Having pre-negotiated, agreed-upon ship criteria helps corporate managers judge for themselves whether or not a product is ready to ship. Pre-negotiated criteria helps the project team decide what product quality is when one no one is stressed from the project work.
It's important for testers to prioritize their testing so that the product ship criteria can be met. Since very few projects have enough time to do everything, getting the testers enough information for what and when to test is a very important role for the test manager.
Corporate managers need to understand the product ship criteria enough to be able to judge whether the product is ready to ship at a given date. I don't believe the test group should be either the holder of product approval or rejection-that role belongs to corporate management. Having pre-negotiated, agreed-upon ship criteria helps corporate managers judge for themselves whether or not a product is ready to ship. Pre-negotiated criteria helps the project team decide what product quality is when one no one is stressed from the project work.
Gather Product Information
The successful test manager also gathers product information, in the form of defect counts, test pass rates, and other meaningful data. The test manager defines the data, and then gathers it for presentation to corporate management. For example, I gather what I consider to be the "basic" metrics:
Defect find and close rates by week, normalized against level of effort (are we finding defects, and can developers keep up with the number found and the ones necessary to fix?)
Number of tests planned, run, passed by week (do we know what we have to test, and are we able to do so?)
Defects found per activity vs. total defects found (which activities find the most defects?)
Schedule estimates vs. actuals (will we make the dates, and how well do we estimate?)
People on the project, planned vs. actual by week or month (do we have the people we need when we need them?)
Major and minor requirements changes (do we know what we have to do, and does it change?)
Measure what you’ve done
Few days ago, at a testing conference, I met a few of incredibly busy test managers. They were taking conference calls, sending and receiving email, and still trying to attend sessions at the conference. I found that one of the reasons managers are so busy is that some of them don’t know whether they’ve gotten anything done. They’re all crazy-busy, but most of them feel as if they’ve accomplished nothing.
When your efforts have more to do with team performance and avoiding disasters than a tangible product, how can you measure what you’ve done in a concrete way? If you are a test manager, here are some ways to measure what you’ve done:
How many people have you unwedged or unblocked this week?
Sometimes our staff may not realize they are stuck trying to solve problems. If you are able to recognize that someone is stuck and point them in a more useful direction, then you’ve accomplished valuable work.
How many of your staff were able to get more work done this week, based on what you said?
Managers leverage the work other people do. When you facilitate other people’s work, then you’re doing management work. If you discussed test plans or your staff’s choices about which tests to implement, then you’ve helped people leverage their work.
How much strategic work did you do this week?
Managers set the strategy for their groups, and then put the tactics in place to make it happen. Maybe you’re working on improving your test capability with a new test lab, or arranging ongoing education for your testers. Maybe you’re working on decreasing time to market by changing how you plan your projects, or how the developers and testers work together. Strategic goals are long-term goals, so you’d expect to make only a little progress on strategic work each week. But the effort you invest and the progress you make certainly count.
How many crises did you prevent?
Every time you solve a problem and prevent a disaster or crisis, you’ve allowed your technical staff to go forward. For example, if you realized early enough in the project that the developers and testers were working toward different goals, and you started developing release criteria, you’ve averted plenty of problems and a potential ship crisis.
How many meetings were you able to cancel?
Unfortunately, too many of us work in meeting-happy organizations. A meeting that doesn’t include the following characteristics is not likely to help your project or your team move forward:
It starts on time
It ends on time or early
You can leave the meeting if you’re not needed
Action items come out of the meeting
The meeting has an agenda and minutes
If you are able to cancel meetings that don’t actually help move a project forward, then you’ve done wonderful management work. Look for these opportunities and take advantage of them when you see them.
These are just five ways to measure your managerial output. You may want to create a short checklist that you review each week to keep track of your efforts. Any work you do that helps your staff and the long-term health of your organization is good, tangible work. Count it.
When your efforts have more to do with team performance and avoiding disasters than a tangible product, how can you measure what you’ve done in a concrete way? If you are a test manager, here are some ways to measure what you’ve done:
How many people have you unwedged or unblocked this week?
Sometimes our staff may not realize they are stuck trying to solve problems. If you are able to recognize that someone is stuck and point them in a more useful direction, then you’ve accomplished valuable work.
How many of your staff were able to get more work done this week, based on what you said?
Managers leverage the work other people do. When you facilitate other people’s work, then you’re doing management work. If you discussed test plans or your staff’s choices about which tests to implement, then you’ve helped people leverage their work.
How much strategic work did you do this week?
Managers set the strategy for their groups, and then put the tactics in place to make it happen. Maybe you’re working on improving your test capability with a new test lab, or arranging ongoing education for your testers. Maybe you’re working on decreasing time to market by changing how you plan your projects, or how the developers and testers work together. Strategic goals are long-term goals, so you’d expect to make only a little progress on strategic work each week. But the effort you invest and the progress you make certainly count.
How many crises did you prevent?
Every time you solve a problem and prevent a disaster or crisis, you’ve allowed your technical staff to go forward. For example, if you realized early enough in the project that the developers and testers were working toward different goals, and you started developing release criteria, you’ve averted plenty of problems and a potential ship crisis.
How many meetings were you able to cancel?
Unfortunately, too many of us work in meeting-happy organizations. A meeting that doesn’t include the following characteristics is not likely to help your project or your team move forward:
It starts on time
It ends on time or early
You can leave the meeting if you’re not needed
Action items come out of the meeting
The meeting has an agenda and minutes
If you are able to cancel meetings that don’t actually help move a project forward, then you’ve done wonderful management work. Look for these opportunities and take advantage of them when you see them.
These are just five ways to measure your managerial output. You may want to create a short checklist that you review each week to keep track of your efforts. Any work you do that helps your staff and the long-term health of your organization is good, tangible work. Count it.
Subscribe to:
Posts (Atom)
