Manage Learn to apply best practices and optimize your operations.

Enterprise network testing: Best practices for developing a test lab

Enterprise network testing can minimize IT service outages, yet developing a test lab can bring up various issues. Knowing best practices for running a test facility is imperative.

Applying the theory of enterprise network testing to help minimize IT service outages and see how testing fits into the overall enterprise architectural strategy of an organization is all fine and well. However, enterprise network testing can serve to be challenging when applied to a real business with a closely monitored budget for IT spending. When attempting these tests, it is important to know potential costs associated with them, such as building, staffing and operating a lab. To achieve the most productive and cost-efficient strategy, knowing best practices for creating a network test lab and understanding when outsourcing test functions is most appropriate is key.

This chapter, Testing and Lab Strategy Development, from the book Enterprise Network Testing: Testing Throughout the Network Lifecycle to Maximize Availability and Performance from Cisco Press, covers these areas by focusing on business cost analysis of a test organization startup, a description of different funding models, test lab facility designs and details on network test lab operations.

Cost analysis and resource planning for enterprise network testing

The first step in planning for enterprise network testing is to consider the costs and potential risks associated with adopting a test exercise. As business costs are examined in terms of capital expenditures (CAPEX) and operational expenditures (OPEX), a business’ preference to use capital versus operational funds may be influenced by the current economic and business climate. 

The initial establishment of a medium- to large-sized test lab can serve to be quite costly and have a high CAPEX effect. Also, the decision of “build versus buy” can also be made regarding enterprise network testing, as testing can be done in-house or outsourced to a test services provider.

When evaluating the startup of a new test facility, it is also imperative to review environmental considerations, as they can vary significantly. The following areas should be considered regarding environmental factors:

  • Physical space
    • Dedicating a separate, distinct, controlled-access area in the test facility is vital to ensure that staff is not subject to complications and that production network issues are not compromised. The space considerations must include such items as network equipment, test gear, tools storage, supporting servers and lab infrastructure space.
  • Power
    • Several factors need to be considered regarding power consumption in a network test lab: an understanding of the gear to be deployed within the lab and its power requirements; potential costs in adding conduit capacity, which may relieve cost of additional circuits; and having the power source as close to the lab as possible.
  • Climate control
    • The need for a controlled-climate space will be dictated by the amount and type of equipment in the lab as well as human needs. It is important to factor in the thermal output of equipment in the lab, but also human involvement, as these can potentially affect performance in a test lab.
    • Consider humidity levels and lighting, how airflow will occur between devices, how cold air enters and how excess heat may be vented.
  • Access
    • Controlling physical access to the lab space is essential for minimizing potential interruptions, accommodation for bringing equipment in and out of the environment and how lab staff accesses the equipment locally or remotely.
  • Other infrastructure
    • Other infrastructure aspects relevant to a test facility can be underestimated. It is important to also consider cabling, cabinetry, network connectivity, Unit Under Test (UUT) network devices and antivirus protection.

Thinking in OPEX terms, there are also several expenses in many areas of a network test lab’s operation. Some key points include the following:

  • Staffing is generally the largest operational expense of a test lab. These costs include benefits, training and salaries. It can be beneficial for a test facility to develop a staffing structure, including a senior level staff to lead the operation of the lab. Having a structured staff level, activity and volume will inevitably help to control costs.
  • The power consumption for a lab’s operations will be determined by the equipment in use and what’s needed to control the climate and lighting in the area. Using automated tools for controlling power availability can help to alleviate power consumption and associated costs.
  • As for the physical facility, the cost of the actual space may be a chargeable item, depending on corporate policy. Pay attention to whether the facility is leased or not—if the lab is unmanned, costs can be minimized.
  • It is important to not overlook management and maintenance obligations of the lab. Be sure to consider systems to manage tasks, scheduling of equipment, builds and teardowns, and inventory management. These potential costs will appear as the overhead of keeping the databases’ content current and maintaining internal support needs.
  • Other OPEX considerations include budgeting allowances for small consumables associated with the lab operation, such as screwdrivers and tie-wraps, the need for additional overhead management tools and an equipment management “checkout” tool.

Financial solutions and considerations in enterprise networking testing

With all this cost-effective mentality, financing models are important to ensuring a productive test lab. There are variations on funding approaches, based on size and financial commitment, but either way, they are essential to a test facility. Some approaches to consider once you’ve defined those variances include cost of business, project-based funding, departmental chargeback, testing as a business function and return on investment. All have their own pros and cons, but it’s up to the lab team and staff to determine which is most applicable for the facility.

One way to avoid costs as much as possible is to turn to a test services vendor. Doing so will incur minimal CAPEX costs and r be accounted as OPEX and a reduction in staff workload. While these are great pros, leaving this type of work in the hands of a vendor requires a great deal of trust, which can be risky. Some work may be off the staff’s hands, but you must ensure that you chose the right organization to outsource enterprise network testing activities and that the vendor is doing their part.

The design of the test lab facilities is also important when talking budget. The main idea is to enable the test area to imitate the production environment as much as possible, within cost constraints. At the same time, the physical aspects of the lab facility need attention from a design perspective. To achieve this, lab designers should pay close attention to choosing the most appropriate hardware and software for the facility. They should also focus on square footage, power availability and layout, air conditioning and airflow, cabling, special and free-standing gear and equipment storage.

Finding management and control in the enterprise network testing chaos

A formal test organization also has a charter which defines the operational guidelines. This document helps to outline the lab’s capabilities and business focus, organize relationships with other corporate entities and guide the lab staff. Having this charter may help to alleviate questions from individuals on the lab team. It may also help those who are outside the team but are either helping with building, testing or costs.

With all the moving parts in a test lab—from staff to budget to design to maintenance—it can be overwhelming to manage. But installing one or a few management system(s) at the same time as your lab infrastructure gear and equipment can be very valuable.

Some essential management systems to consider include an equipment inventory system to help with product identification and data for budget purchasing decisions; an equipment scheduling/lab checkout tool to prevent overscheduling of equipment for tests and aid the test engineer in finding the equipment needed for a particular work effort; a team website to present a clear focal point to the rest of the organization and to showcase team efforts; and security and safety management for both the equipment and the staff.

When building and running a successful testing organization, there are many moving parts, but the essential parts include ensuring a proper budget, staff requirements, lab design and test lab operations. When all these elements work together, a productive and successful test lab emerges.

Visit Cisco Press' website for more books and information on network technology.

More on enterprise and network testing

Next Steps

How you can use DevOps for networking testing.

This was last published in June 2011

Dig Deeper on Network management and monitoring