How to manage Quality for a project based on new technology

The project with new technology, holds many problems with itself that you did not anticipate. Many of these problems may require new tools, new skills, or substantial time to solve.

A process that is optimized to handle problems you understand may be wrong for problems you don’t yet understand. That’s why it needs a more exploratory, risk-oriented, incremental form of process, as opposed to one that is emphasizes
pre-planned, predictable tasks.

The key to working in unfamiliar territory is to change your test management focus from task fulfillment to risk management. The project team should continue to devote attention to planning and fulfilling project tasks, but that should not be the focus of the test team. Instead, the test manager must assume that any given prediction, at any given time, may be wrong.

A risk managed testing project implies specific strategies to deal with that assumption. Here’s my list:

· Produce a risk list for the project. If not a printed list, any leader on the team should be able to tell you what the major risks are. All project leaders should be in general agreement about the list.

· The team should practice talking about risk areas, rather than discouraging them as “negative thinking.” The risk that you don’t discuss is the one that you also won’t guard against. However, you should also stress that a new technology project requires you to be bold and to accept a certain amount of risk. The real problem is when there is an inconsistent attitude about risk, across the team.

· There should be some process whereby anyone on the project can learn about the current status of risk areas. There should be frequent project reviews (every few weeks) that involve a level of management one step higher (at least) than day-to-day management.

· There should be a change control process that prevents the careless introduction of new risk drivers after the scope of the project has been set. A clear charter for the project can help in limiting scope creep. Too much change control, too early, can also hurt the project. Consider establishing a progressive change control strategy, whereby the closer the product is to shipping, the stricter changes are controlled.

· Avoid specific promises to upper management about revenue dates, or anything else, unless you are in a position to fulfill those promises without compromising the project. Until the risks of the project have been explored and mitigated, such promises are premature. Also, look for an exit strategy that allows the project to be cancelled if it becomes apparent that the risks are too large relative to the rewards.

· For each risk driver, look for a strategy that will allow that element to be dropped from the project if it turns out to be too difficult to solve the problems that come up. Consider moving especially risky components to a follow-on release, or making them optional to install. I call this an “ejection seat” strategy. If risky components cannot be ejected, they can drag the whole project down.

· Adopt a prototyping strategy, or some other way that risk drivers and risks can be explored while there’s still time to do something about them, or it’s still early enough to cancel the project without huge losses.

· Adopt a test management process that dynamically reallocates test effort to focus on risk areas, as risk areas emerge. It helps to have an explicitly risk-based test plan. Look for strategies that minimize the amount of time between defect creation and resolution.

· The project should have a larger test team than normal, in order to react quickly to new builds. The development team does not have to be larger, in fact many say it should be smaller, but it must have the ideas, motivation, and skills needed to do the job. Special training or outside support should be considered.

· Draft a test plan that includes iteration and synchronization. Think of it like rock climbing or crossing a fast river by jumping from rock to rock: plan ahead, but revise the plan as new challenges arise and new information comes to light. Don’t let the bug list spiral out of control, but rather fix bugs at each stage. You might have to scrap some component of your technology and re-design it, once you have some experience with it in field testing. Don’t be surprised by that. It’s perfectly normal, and it’s the people who refuse to rewrite bad code who suffer more, in the end. Go through a test-triage-bug fix-release end game cycle for each iteration.

· Pay attention to how the product will be tested. Assure that the designers make every reasonable effort to maximize the controllability and visibility of each element of the product. These can include such features as log files, hidden test menus, internal assertion statements, alternative command-line interfaces, etc. Sometimes just a few minutes of consideration for the test process can lead to profound improvements in testing.

· Don’t put all your eggs into the risk-based basket, because you may inadvertently overlook an area that was risky, just because you didn’t anticipate that risk. So, devote a portion of your test strategy, maybe a third, to a variety of test approaches that are not explicitly risk-based. Also, establish a process for encouraging new approaches to be suggested and tried.

· Look for ways of making contact with the field about the emerging product. In speculative projects, problems often emerge that are invisible to the developing organization, yet obvious to customers. Also, assure that technical support, product management, and technical writers are well connected to the test team, so they can feed information to the testers without too undue effort.

· Look for records of schedules, schedule slips, meeting notes, and decisions made. Any metrics or general information that can be collected unobtrusively will help you understand the dynamics of the new technology and the team. That, in turn, helps plan the next project.