This tip is excerpted from "Network troubleshooting and diagnostics," Chapter 4 of The Shortcut Guide to Network Management for the Mid-Market, written by Greg Shields and published by Realtimepublishers.com. You can read the entire
e-book for free at the link above.
A baker can't bake a cake without a good cake pan, and a mechanic won't get very far in fixing your car without a solid wrench set. Though the tools for network administrators are different and more difficult to wrap your hands around than these examples, the need for them is no different. Lacking the appropriate set of tools will usually prevent the job from getting done. This section will discuss some of the tools and their feature sets that you should add to your quiver to support your technical troubleshooting activities.
Telnet and SSH
Telnet, originally short for "TELetype NETwork" and now considered a proper name all to itself, is the most common mechanism for forwarding a system's command-line console session to a remote host. Telnet is entirely textual and command-line driven, which makes its use difficult for newer administrators. Telnet is used by virtually all UNIX hosts as well as network devices for device configuration and administration.
SSH or "Secure SHell" is a similar protocol intended to accomplish the same goal as Telnet but with an element of built-in security. SSH uses public-key cryptography to authenticate a user to the system as well as provide confidentiality and integrity of data passing between the SSH client and server. SSH is quickly becoming the standard for remote terminal applications due to this added security built-in to its protocol.
For either protocol, the necessary tool in your troubleshooting quiver will be a Telnet or SSH client. Numerous clients exist, and some have more features than others. Some features you may want to consider when looking for a good Telnet or SSH client are:
- Text colorization
- Function key mapping
- Remote file copying support
- Server connection profiling
- Alarm generation
- Script recording and playback
- Session tabbing
- Secure password caching
|As a rule, always try to use SSH over Telnet when it is supported by your network devices. Telnet sends data and passwords across the network in clear-text, which allows an attacker to easily sniff the traffic as it traverses the network. This is especially true when connecting to devices across the Internet.|
Serial port tools
Although a good Telnet or SSH client will help you connect to already-configured network devices, these devices often must be initially configured using an on-board serial port before they can connect to a network. The on-board serial port includes a cable transceiver that converts the network device's serial port to one that is useable by a desktop or laptop system. To connect the desktop or laptop system to the network device, a serial port tool is needed.
Like Telnet/SSH clients, serial port tools come in many flavors. As an example, one very basic serial port tool, HyperTerminal, has been available with Microsoft Windows systems from the time of Windows 95 up until the release of Windows Vista. However, because network administrators make substantial use of these tools in network setup and troubleshooting, there are additional feature sets above those in the native tools that are necessary to ease administration. Some features you may want in a good serial port tool are:
- Rich copy-and-paste
- Multiple terminal emulation support
- Printing and print selection
- Automation and scripting
- Text-to-file exporting
- Extended serial support conversion
Network monitoring tools can either be a component of your NMS or a separate utility. In either case, a network monitoring tool is used to record and analyze the characteristics within its configured network. Network monitoring tools can monitor for network performance as well as network outage and device resource use. They typically aggregate multiple network devices into a single user interface for cross-device analysis. Some features in a network monitoring tool that are critical for the troubleshooting process are:
- Multiple device capability
- Traffic graphing support
- Device resource use monitoring
- Alerting and notification via multiple mediums
- SMS/text messaging support
- SNMP management
- Traffic analysis
- Built-in traffic filters and aggregators
Knowing what is going on within your network is only useful if you're aware of all the devices that make up that network. If a problem on the network occurs because of a rogue device, it is often difficult to track down that device without a tool to do the tracking. Network discovery tools are those that scan the network for known device heuristics. When a device heuristic is found at a particular address, the network discovery tool logs the location and its believed device type, then reports that information to the administrator.
Numerous network discovery tools exist and each has a specific mechanism for seeking out devices -- by IP address, MAC address, SNMP response, DNS entry, or even individual switch port on switching devices. Some features useful in a network discovery tool are:
- NMS integration
- Multiple IP range entry
- Fast scanning
- Device heuristic databases with SNMP
- Switch port mapping
- Data export to common file formats
Figure 4.3: A typical network discovery tool will scan a range of addresses to look for the presence or absence of a connected device. Some network discovery tools can compare results with known devices to look for rogue devices on the network.
Attack identification and simulation
Administrators unfamiliar with the changes in a network's functionality during an external attack situation will be unprepared for fending off that attack once it occurs. Attack identification and simulation tools enable the administrator to identify when common network attacks occur such as broadcast storms, cache poisoning, replay attacks, and so on. They also allow for the simulation of such attacks upon a network to monitor and analyze the behavior of that network as well as to assist in preparing the network against a real attack by an outside attacker.
Attack identification tools such as network intrusion detection systems and network intrusion protection systems can be complicated to install and manage due to the prevalence of false positives and false negatives such systems can generate. The following list highlights features of interest in either type of tool:
- Performance monitoring elements
- Identification databases with real-time update
- Multiple attack profiles
- Dictionary and brute force capabilities
- Network device security checks
- Port scanning
- Network jamming
- Remote TCP resetting
|Attack simulation tools should be kept out of the hands of unprepared administrators, as such tools have the capability of inhibiting the successful operation of the network.|
We've talked about SNMP and SNMP traps before within this guide, but SNMP trapping tools have a different use than those in your NMS. SNMP trap receiving tools are out-of-band tools that can receive, analyze, and display low-level trap information from an SNMP-enabled device for purposes of troubleshooting and SNMP analysis outside the NMS. SNMP trap editing tools allow for the editing of trap templates to customize NMS response when traps occur. These tools incorporate some needed features for advanced SNMP manipulation:
- Data export to common file formats
- Trap manipulation
- Tree view
- Trap mimicking and simulation
Ping, Traceroute, and ARP
Although ping, traceroute, and ARP commands are available in virtually every OS in existence, the tools present natively in these OSs often involve minimal functionality. Additionally, they typically only allow for result output to the screen, lacking the ability to natively capture results into a more useable format.
For network administrators who regularly use these tools, the additional functionality of non-native variants of them may be useful for the troubleshooting process. Consider these added functions when looking for replacements for these tools:
- Enhanced ping timing response
- Data export to common file formats
- Graphical response representation
- Multiple, simultaneous host support
- IP address range support
- Enhanced traceroute result information
- Remote ping sourcing
As explained in Chapter 1, Management Information Bases (MIBs) are databases of characteristics about network devices. Those databases are released by the manufacturer and house readable and writeable information about the configuration and status of the network device. A MIB Browser is a specialized tool that can peer into the data inside a MIB and pull out relevant Object ID (OID) information. Remember that OIDs are little more than strings of numbers used as unique addresses for device data. A good MIB Browser will include a pre-populated database of known OIDs and their related data. It will also enable the ability to "walk the MIB tree," gathering all known data from that MIB and presenting it to the administrator.
The real power of an effective MIB Browser is in its ability to view and search the MIB for relevant information and allow the administrator to modify and customize that information as necessary. A good MIB Browser will typically include this functionality:
- Remote device support
- Large database of known OIDs
- View/search/walk via tree-view
- Editing functions
- Reading/writing support
- Multiple-device support
|MIB Browsers are primarily used as customization tools for the SNMP-enabled devices plugged into your NMS.|
NETWORK TROUBLESHOOTING AND DIAGNOSTICS
Developing good troubleshooting technique
Tool suites for identifying the problem
IP address management
Network engineering applications
About the author:
Greg Shields is a Principal Consultant with 3t Systems in Denver, Colorado - www.3tsystems.com. With more than 10 years of experience in information technology, Greg has developed extensive experience in systems administration, engineering, and architecture specializing in Microsoft, Citrix, and VMware technologies. Greg is a Contributing Editor for both Redmond Magazine and Microsoft Certified Professional Magazine, authoring two regular columns along with numerous feature articles, webcasts, and white papers. He is known for his abilities to relate highly technical concepts with a drive towards fulfilling business needs. Greg is also a highly sought-after instructor and speaker, teaching system and network troubleshooting curriculum for TechMentor Events, a twice-annual IT conference, and producing computer-based training curriculum for CBT Nuggets on numerous topics. Greg is a triple Microsoft Certified Systems Engineer (MCSE) with security specialization and a Certified Citrix Enterprise Administrator (CCEA).
This was first published in September 2007