Test environment management (TEM) is a function in a software delivery process which aids the software testing cycle by providing a validated, stable and usable test environment to execute the test scenarios or replicate bugs.

Background

As with a scientific experiment, in Testing repeatability and control of variables is essential. In testing a key component of this control is to manage the environment in which testing is taking place. This environment specifically includes the underlying hardware and software which supports the actual software under test. This encompasses items such as servers, operating systems, communications tools, databases, cloud ecosystems browsers.

In early testing stages only limited formal management of environments is required, if any. For example programmers may typically perform their testing within standardised IDEs which provide control by default. However at later stages, test execution will tend to work across multiple technologies and development streams, and typically involving multiple (teams of) testers. In these circumstances individual testers cannot reasonably be expected to exercise control over the technical landscape. This is where the need for some formal Test Environment Management function arises.

Activities

The activities under the TEM function include:

• Maintaining a central repository of test-environments in scope with their latest version and connectivity details (Information management)
• Allocation of test environments (booking/scheduling) to teams as per requirement. (Demand management)
• Creation of new test environments as per requirement. (Supply management)
• Environment Monitoring (Monitoring)
• Deleting/ updating outdated test-environments and its details (Housekeeping)
• Preliminary investigation of issues on the environment and sometimes co-ordination till an issue resolution (Incident Management)
• Analyzing data for environment issues, identifying trends and taking pro-active steps to resolve issues / co-coordinating for a long term fix. (Problem Management)
• Test data management to ensure that test data is available to testers when needed via a TDM tool or test data refresh from other environments
• Status Accounting to report on service, usage, and availability metrics to provide full transparency into test environment management activities
• Continuous improvement to continuously evaluate test environment management services for improvement opportunities (people, process, and tools)
• Automation to eliminate manual tasks as much as possible to improve efficiency

Tools

1. Configuration management database software : This tool would be required to maintain a repository of the environment components and its versions. The data in this tool would also be helpful in incident management and problem management.
2. Booking tool : This tool would be required to capture the allocation of test environments and to check for the availability of the environments. Usage analysis can also be captured in it.
3. Problem / incident management tools : This tool would be used to capture the problem / incident data and to manage the life-cycle of the incident / problem. Reports generated based on this data would give good insights into the health of the test environments.
4. Test data management tool: This tool allows testers to create, recycle, mask, and use their test data on demand.

Many teams use spreadsheets instead of using specific tools for the first two areas if the data is less. However, if the data is more, it is recommended to use specialized tools for it.

Overheads

• Cost of maintaining a separate team for environment management would need to be justified.
• A lot of co-ordination work would need to be done by the TEMs proactively to ensure they have the latest updates available.