Problem solve Get help with specific problems with your technologies, process and projects.

Standardizing performance measurement procedures

Dr. Jorgenson,
I'm trying to benchmark data throughput on a Nortel 10/100 switch. I'm trying to confirm throughput testing when rate limiting is applied on the switch. I have a lab environment with a Nortel PP8600 switch, one network laptop with a 3rd party FTP/TFTP server S/W, one network laptop with FTP/TFTP client S/W and SnifferPro software.

I'm looking for some guidelines or some industry standard testing (telecommunications) to help guide me to what type of testing is the most valid and how to interrupt my results.

Any help that you can provide would be very helpful. Thanks Marc
Marc, there are some efforts made at standardizing performance measurement procedures. But I don't know of anything that I'd recommend for your particular objective. Perhaps someone can enlighten us both.

However, permit me to comment on what I think you need to watch for and suggest an effective solution given the situation you have described. The switch utilizes a CISCO CAR-style mechanism, dropping packets when rates of data transfer exceed the specific limits. This mechanism implicitly counts on TCP's congestion avoidance to moderate the amount of data being transmitted. Any attempt to measure the effective rate limiting may be obscured by the response of the TCP stack.

Using a standard FTP server and client is not likely to give you a clear measure of the limiting. In fact, using TCP to establish your baseline is likely to give you an erroneous impression of the rate limiting. Eventually you might like to get a relative measure of the rate a typical TCP-based connection will be limited to. However, don't start there.

I suggest the following:

  1. Get the performance measurement tool, iPerf, from NLANR at http://dast.nlanr.net/. It is an active flooding tool that can make end-to-end data transfer tests much more reliable - it is the poor man's SmartBits. It works in both TCP and UDP modes.
  2. Use two relatively high-performance workstations (1+ GHz with 512 Mbytes memory and good NICs like the 3Com 905 b/c) - the portable may give you decent performance but it is less likely. And if the portable's NIC is PCMCIA-based, forget it.

    However, if you have no choice in the matter, be warned that the portables' NICs and busses may be doing the all the rate-limiting depending on the rate that the switch is set for.

  3. Hand set all interfaces to full duplex mode - do not rely on auto-negotiation. Never rely on auto-negotiation. Ever. For anything.
  4. Baseline the performance with iPerf in UDP mode between the two end stations first and then add variables as you go. Check for the maximum rated transfer, watching that packet loss rates are stable, and establish the stable rates below some minimum packet loss.
    1. without the switch in between (i.e. back-to-back with a cross-over cable.)
    2. with the switch but without the rate-limiting
    3. with switch and rate-limiting
    4. varying the rate limiting
  5. Try iPerf in TCP-mode for comparison
  6. Try using FTP clients for comparison ? however, if FTP transfer is critical as a transfer mechanism, consider getting a robust parallel connection FTP client like gridFTP, otherwise you are unlikely to stress the link or the rate-limiting very hard.

If this doesn't get you a clear picture of the switch performance, you tell me.

This was last published in February 2003

Dig Deeper on Network management and monitoring

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.