Reengineer The Test Management

Most organizations don’t have a standard process for defining, organizing, managing, and documenting their testing efforts. Often testing is conducted as an ad hoc activity, and it changes with every new project. Without a standard foundation for test planning, development, execution, and defect tracking, testing efforts are nonrepeatable, nonreusable, and difficult to measure.
Generating a test status report is very time consuming and many times not reliable. It is difficult to procure testing information such as
How much testing needs to be done?
How much testing has been completed to date?
What are the results of these tests?
Who tested what, and when was it last tested?
What is the defect number for this failed test case?
Do we have test results for this build?
Do we have a history of test case results across different builds?
How can we share the test cases with a remote team?
Does our product meet the requirements that we originally set?
What is the requirements test coverage?
Is our product ready for release?
Getting this information fast is critical for software product and process quality. But many times, it is difficult to get this information, depending on the way test cases and execution results are defined, organized, and managed.
Most organizations still use word processing tools or spreadsheets to define and manage test cases. There are many problems associated with defining and storing test cases in decentralized documents:
1. Tracking. Testing is a repetitive task. Once a test case has been defined, it should be reusable until the application is changed. Unstructured testing without following any standard process can result in creation of tests, designs, and plans that are not repeatable and cannot be reused for future iterations of the test. It is difficult to locate and track decentralized test documents.
2. Reuse. Because it is difficult to locate test cases for execution, they are seldom used in day to day testing execution.
3. Duplication of test cases and efforts. It is difficult to locate a test case, so there are chances of duplicating the same and wasting the testing effort.
4. Version control. Since there is no central repository, version control becomes difficult, and individual team members may use different versions of test cases.
5. Changes and maintenance. Changes to product features can happen many times during a product development lifecycle. In such scenarios, test cases can become obsolete, rendering the whole effort in test planning a fruitless exercise. It is important to keep the test case list updated to reflect changes to product features; otherwise, for the next phase of testing there will be a tendency to discard the current test case list and start over again.
6. Execution results—logging and tracking. The test execution result history is difficult to maintain. It is difficult to know what testing has been done, which test cases have been executed, results of each test case that is executed, and if a problem report has been written against this failed test case.
7. Incomplete and inconsistent information for decision-making. A defect database provides only one side of the information necessary for knowing the quality of a product. It tells what is broken in a product, and what has been fixed. It does not tell what has been tested and what works. This is almost as important as the defect information.
8. Test metrics. If we cannot have a history of test results, it is difficult to generate test metrics like functional test coverage, defect detection effectiveness, test execution progress, etc.
9. Difficult to associate related information. We also need to deal with huge quantities of information (documents, test data, images, test cases, test plans, results, staffing, timelines, priorities, etc.).
10. Nonpreservation of testware. It is essential that testware (test plans, test cases, test data, test results, and execution details) be stored and preserved for reuse on subsequent versions of a single application or sharing between applications. Not only does this testware save time, but over a period of time it gives the organization a pattern, knowledge base, and maturity to pinpoint the error-prone areas in the code development cycle, fix them, and prevent the errors from recurring.
11. Inconsistent processes. Organizations are not static. People move from project to project. If the testing job is performed differently for each project or assignment, the knowledge gained on one assignment is not transferable to the next. Time is then lost on each assignment as the process is redefined.
12. Requirements traceability and coverage. In the ideal world, each requirement must be tested at least once, and some requirements will be tested several times. With decentralized documents, it is difficult to cross-link test cases with requirements. Test efforts are ineffective if we have thousands of test cases but don’t know which one tests which requirement.

The problems due to unstructured, decentralized test management can be solved by reengineering the test management process. A testing project starts by building a test plan and proceeds to creating test cases, implementing test scripts, executing tests, and evaluating and reporting on results.
The objectives of reengineering test management are to
1. assist in defining, managing, maintaining, and archiving testware
2. assist in test execution and maintaining the results log over different test runs and builds
3. centralize all testing documentation, information, and access
4. enable test case reuse
5. provide detailed and summarized information about the testing status for decision support
6. improve tester productivity
7. track test cases and their relationship with requirements and product defects
A Reengineered test management process can help in improving key processes in test definition, tracking, execution, and reporting.