Recommendations for better Test management

The following are general recommendations that can improve software test management.

Start test management activities early

While this may seem like the most obvious suggestion, few software projects truly apply this concept. It is easy and not uncommon to begin identification of test resources at an early stage. However, many test analysis activities (such as the identification of critical, high-priority test cases) can and should begin as soon as possible. As soon as use cases are developed enough to have a flow of events, then test procedures can be derived. If a project is not utilizing use case requirements, then tests can still be derived from the validation of initial requirements. Developing tests as soon as possible helps alleviate the inevitably forthcoming time constraints.

Test iteratively

Software testing should be an iterative process, one that produces valuable testing artifacts and results early on in the overall project lifecycle. This follows the first recommendation of starting the testing process early: an iterative testing process forces early attention to test management. Test management guides this by organizing the various artifacts and resources to iterations. This risk-based approach helps ensure that changes, delays, and other unforeseen obstacles that may come up in the project timeline can be dealt with in the most effective manner.

Reuse test artifacts

Reusing test artifacts within a project, or across projects, can greatly improve the efficiency of a testing team. This can greatly relieve the pressure of limited time and limited resources. Reusable artifacts include not only test automation objects, but also test procedures and other planning information. In order to reuse artifacts efficiently, test management must do a good job of organizing and delineating the various testing-related information used for a given project. Reuse always requires some forethought when creating artifacts, and this principle can be applied in test management generally.

Utilize requirements-based testing

Testing can be broken down into two general approaches:

• Validating that something does what it is supposed to do
• Trying to find out what can cause something to break

While the latter exploratory testing is important to discover hard-to-predict scenarios and situations that lead to errors, validating the base requirements is perhaps the most critical assessment a testing team performs.

Requirements-based testing is the primary way of validating an application or system, and it applies to both traditional and use case requirements. Requirements-based testing tends to be less subjective than exploratory testing, and it can provide other benefits as well. Other parts of the software development team may question or even condemn results from exploratory testing, but they cannot dispute carefully developed tests that directly validate requirements. Another advantage is that it can be easier to calculate the testing effort required (as opposed to exploratory testing, which is often only bounded by available time).

It can also provide various statistics that may be useful quality metrics, such as test coverage. Deriving test cases from requirements, and more importantly tracking those relationships as things change, can be a complex task that should be handled through tooling. RequisitePro and the test management capabilities in ClearQuest provide an out-of-the-box solution that addresses this need.

The constraint to this process is that it depends on good system requirements and a sound requirements management plan to be highly effective. Exploratory testing, on the other hand, can be more ad hoc. It relies less on other parts of the software development team, and this can sometimes lead to testing efforts focusing less on the important tasks of validating requirements. While a superior testing effort should include a mix of different approaches, requirements-based testing should not be ignored.

Leverage remote testing resources

To help alleviate resource shortages, or to simply maximize the utilization of personnel, you should leverage whatever resources you can, whereever they are located. These days resources are likely to exist in multiple geographic locations, often on different continents. This requires careful and effective coordination to make the most of the far-flung testers and other people involved with test management. There can be considerable technical challenges for this to be efficient, and therefore proper tooling is needed. The test management capabilities in ClearQuest with MultiSite is a tool that simplifies the complexities of geographically distributed test coordination.

Should you utilize a Web client or automatically replicated data? These are two solutions available that make collaboration with remote practitioners possible. The former is simple and relatively easy, but there is still a potential constraint of network latency, especially if accessed across the globe. For remote access by a limited number of people or with limited functionality, this is a good solution. However, for situations where a number of people in different locations make up an overall virtual testing team, you will need to have data copied on local servers to maximize the speed at which they can work. This also means that you will need an easy and seamless way to automatically synchronize the data across each location. This is where ClearQuest MultiSite can be essential for test management.

Defining and enforcing a flexible testing process

A good, repeatable process can help you understand a project's current status and, by being more predictable, where it's headed. However, different projects will have different specific needs for the testing effort, so a test management process that automates workflows needs to be flexible and customizable. The process should be repeatable (to provide predictability), but more importantly, it must allow for improvements. It has to be easy enough to make revisions, including adjustments during the course of an iterative project, so that it can be optimized through changing needs.

Defining a process with workflows to guide team members doesn't do much good if it can't be enforced in any way. How strongly it needs to be enforced will vary with different organizations and projects. Software projects in many organizations now have a need to comply with various regulations, such as SOX and HIPPA. Some have a need for auditability of changes, project history, and other strict compliance validation such as e-signatures. Whether your project's test management requires strict process enforcement or a more casual approach, you need a mechanism for defining and enforcing something. One such test management tool that provides all of these capabilities is the test management capabilities in ClearQuest.

Coordinate and integrate with the rest of development

Software testing has traditionally been kept highly separated from the rest of development. Part of this comes from the valid need to keep the assessment unbiased and increase the odds of finding defects that development may have missed. This need is especially apparent with acceptance testing, where the best testers are ones who are blind to the design and implementation factors. However, this specific need only represents one of many aspects of software testing, and it should not create the barrier and impediments to developing quality software that it usually winds up doing.

Software testing must be integrated with the other parts of software development, especially disciplines such as requirements management and change management. This includes vital collaboration between the different process roles and activities, maximum communication of important information, and integrated tooling to support this. Without this coordination, quality will be reduced from missed or misunderstood requirements, untested code, missed defects, and a lack of information about the actual software quality level.

Communicate status

An effort is only as valuable as it is perceived to be, and how it is perceived depends on what is communicated to the stakeholders. Good test management must provide complete and proper reporting of all pertinent information. Ongoing real-time status, measurement of goals, and results should be made available, in all the appropriate formats, to all relevant team members on the software development project.

Reporting should also be more than just traditional static documents. Given the constant changes going on, it is necessary to have easily updatable output in a variety of formats to properly communicate information. All of this will enable different project roles to make the right decisions on how to react to changes as the project progresses.

Information from the different software disciplines is not entirely separated. This article has already mentioned the important relationships between test management and other disciplines such as requirements, change and configuration management, and development. It is therefore crucial that the outputs coming from test management can be easily combined with other project data. Current technology makes possible consolidated views combining all project metrics into a dashboard so that the overall project health can be determined. Tools also make it possible to clearly show and assess the relationships between test, development, and other project artifacts.

Focus on goals and results

Decide on quality goals for the project and determine how they might be effectively and accurately measured. Test management is where the goals, the metrics used to measure such goals, and how the data for them will be collected are defined. Many tasks in testing may not have obvious completion criteria. Defining specific outputs and measures of ongoing progress and changes will more accurately define the activities and tasks of the testing effort. Keeping specific goals and metrics for testing in mind not only helps track status and results, but also avoids the last-second scramble to pull together necessary reports.

Storing test management results in a single, common repository or database will ensure that they can be analyzed and used more easily. This also facilitates version control of artifacts (including results), which will prevent problems with out-of-date or invalid information. All of this will help project members to understand the progress made, and to make decisions based on the results of the testing effort.

Automate to save time

There is a lot to test management, and its many tasks can be very time consuming. To help save time, tools can be used to automate, or at least partially automate, many tasks. While simple tools like word processors and spreadsheets provide great flexibility, specialized test automation tools are much more focused and provide a greater time-saving benefit. Tasks that benefit greatly from automation include:

• Tracking the relationship of testing to requirements and other test motivators
• Test case organization and reuse
• Documentation and organization of test configurations
• Planning and coordination of test execution across multiple builds and applications
• Calculating test coverage
• Various reporting tasks

Proper tooling and automation of the right tasks in test management will greatly improve its value and benefits.