Network Device Management

This chapter excerpt focuses on device management and the role it plays in network configuration management.

This Content Component encountered an error

This text is excerpted from the eBook Tips and Tricks Guide to Network Configuration Management, Chapter 6: Network Device Management

The book from which this chapter is excerpted presents tips and tricks for four network configuration management topics. For ease of use, the questions and their solutions are divided into sections based on topic, and each question is numbered based on the topic, including Topic 1: Change Management Best Practices, Topic 2: Network Management Security, Topic 3: Network Management Troubleshooting, Topic 4: Change Management Techniques, Topic 5: Selecting and Deploying a Network Device Management Solution, and Topic 6: Enterprise Network Device Management.

To download/read the eBook in its entirety, visit: http://www.alterpoint.com/ebook



Topic 6: Enterprise Network Device Management

Q 6.1: Our enterprise has thousands of network devices and we’re always adding new ones. How can we ensure that a change management solution will accommodate future devices?

A: A little bit of common sense and some investigation will generally provide a satisfactory answer to this question. What you need to do is evaluate each solution for its current breadth of device support, check up on how the solution is architected to accept changes, and make some educated guesses about how the solution will grow in the future.

Intended for Broad Support

On the common sense side, look at the ties each change management solution has. Take CiscoWorks, for example, which is made by Cisco. It’s a good bet that any future Cisco equipment will be addressed by an update or add-on to CiscoWorks. It’s also a good bet that any future Nortel equipment won’t be handled at all. So, if you’re truly in an all-Cisco shop, CiscoWorks might present an acceptable path for future support.

Built for Device Expansion

On the investigation side, find out how each solution provides for future device support. Some solutions, for example, simply ship a new version of the product (upgrading you from 2.1 to 2.2) when a sufficient number of new devices are ready to be supported. Other solutions ship “packs” that add devices to the solution’s list of supported devices. In general, the “pack” idea is preferable because the solution vendor can more easily provide support for one or two new devices rather than waiting until they’ve got enough to justify the release of a new version.

Evaluate the cost of device support, too. Some vendors ship base device support with their software and charge extra to provide support for other devices. Other vendors might offer additional device support on demand as part of a product maintenance agreement. Still others might simply accept requests for device support and post popularly requested device support for free on their Web sites.

Built for Long-Term Change

A larger, and more difficult to evaluate, question is how a change management solution will support radical new management technologies that come out in the future. Without question, every solution on the market today could be modified to support just about anything that comes along. It’s the degree of modification that you should question. For example, suppose somebody invents BTEMP, the Better Than Ever Management Protocol. Device manufacturers universally jump on the BTEMP bandwagon and build it into their latest operating system (OS) updates. Now it’s time to upgrade your management solution to take advantage of BTEMP’s better security, lower bandwidth, or whatever. Some solutions will require a complete version upgrade, meaning you’ll have to wait a while for it to be available, and then likely will have to deal with the bugs often associated with major changes in a software package. Other solutions might simply provide an add-on pack that takes advantage of BTEMP without modifying the base software.

You can get a feel for a solution’s long-term change philosophy by looking at its current breadth of support. For example, a solution built entirely around Trivial File Transfer Protocol (TFTP) might not adapt well to entirely different technologies, although it could be built to easily accommodate new TFTP-based network devices. In contrast, a solution that already supports a broad variety of management technologies—TFTP, HyperText Transfer Protocol (HTTP), Simple Network Management Protocol (SNMP), or even proprietary management interfaces—is more likely to handle radical new management technologies without pain.

Evaluating Solutions’ Growth Potential

As a part of your solution evaluation process, put together a grid that addresses future growth. Table 6.1 provides an example of such a grid.

Criteria Product A Product B Product C
Existing breadth of support Broad—12+ vendors Moderate—8 vendors; mostly higher-end devices Narrow—Vendor specific
Acceptance for new devices Very good—Downloadable support packs Very good—Downloadable support packs OK—Ships a new version to support new device generations
Cost for new devices Popular requests are added to the download site for free; moderate charges for immediate custom add-ons Requires yearly maintenance agreement for popular requests; additional fees for custom add-ons Included with new version; minor upgrades are free, major version upgrades have a cost
Major changes Good—Already supports a broad range of management techniques, implying that support packs contain most of the device interface logic Poor—Only supports TFTP/SNMP; manufacturer states that other techniques would require base code changes; device interface logic is all within the main program, not the support packs Provides full support for vendor’s existing hardware; will provide immediate support for any changes in vendor’s management techniques

Table 6.1: Grid to evaluate solutions’ growth potential.

With a chart like this one, you can easily evaluate solutions’ growth capabilities against your specific needs and concerns. For example, if you’re in a company that only uses network devices from one vendor, than that vendor’s product (Product C in the chart that Table 6.1 shows) would be a great solution. It’s likely that the product will always parallel the vendor’s device technologies, providing a reasonable means of managing those devices well into the future.

Q 6.2: We have hundreds of network devices, so manually retrieving configurations via Trivial File Transfer Protocol just isn’t an option. What are our alternatives?

A: Trivial File Transfer Protocol (TFTP) is great for small environments, but like so many manual solutions, it doesn’t scale well. My preference in a large environment is to select a network device change management solution. Such solutions usually rely heavily on TFTP and other common protocols to retrieve device configurations, but automate the entire process so that dealing with hundreds of devices doesn’t require a team of dedicated people working ’round the clock.

You can, of course, stick with TFTP if you find some way to automate it. Listing 6.1 shows an example Linux script that can use TFTP to pull configurations from either Cisco or Ascend routers (this example is adapted from a more complete script that is available at http://www.securiteam.com/exploits/5RP0E000AA.html).

#!/bin/sh
# grabrtrconf:
# by: Eric Monti 11/1997

TFTPLISTEN="true"
DIR=/tftpboot #might want to use something else
WAIT=6
INT=ppp0

test "$4" = "" && echo "Usage: `basename $0` target _
  write-community tftphost filename [type]" && exit 1
TYPE=$5
test "$5" = "" && TYPE="cisco"

IPADDR=$3
test "$IPADDR" = "." && IPADDR=`/sbin/ifconfig $INT | _
  grep inet | sed "s/:/ /" | awk '{print $3}'`

echo $3

if [ -n $TFTPLISTEN ];then
echo "tftp dgram udp wait root /usr/sbin/in.tftpd _
  in.tftpd $DIR" > /tmp/ind.conf
/usr/sbin/inetd -d /tmp/ind.conf &
rm /tmp/ind.conf
rm -f $DIR/$4
touch $DIR/$4
chmod 666 $DIR/$4
fi

#CISCO get config
test "$TYPE" = "cisco" &&
snmpset -r 3 -t 3 $1 $2 .1.3.6.1.4.1.9.2.1.55.$IPADDR s $4
#ASCEND get config
if [ "$TYPE" = "ascend" ];then
  snmpset -r 3 -t 3 $1 $2 .1.3.6.1.4.1.529.9.5.3.0 a $IPADDR
  snmpset -r 3 -t 3 $1 $2 .1.3.6.1.4.1.529.9.5.4.0 s $4
  snmpset -r 3 $1 $2 .1.3.6.1.4.1.529.9.5.1.0 i 3
  snmpset -r 3 $1 $2 .1.3.6.1.4.1.529.9.5.3.0 a "0.0.0.0"
  snmpset -r 3 $1 $2 .1.3.6.1.4.1.529.9.5.4.0 s ""
fi

sleep $WAIT

if (test `pidof in.tftpd`);then
  echo Receiving file:
  while (test "`pidof in.tftpd`");do
echo -n .
sleep 1
  done
  echo
  echo Transfer Complete

fi

if [ -n $TFTPLISTEN ];then
kill `cat /var/run/inetd.pid`

fi

Listing 6.1: A sample Linux script that can use TFTP to pull configurations.

>> Some of the lines of code in Listing 6.1 were too long to fit on one line in this format; the line continuation character (_) was used to break up lines of code that were too long. You should type these lines of code all on one line, without the underscore.

Now, if you’re perceptive, you’ll have noticed the URL to which I referenced this script: http://www.securiteam.com. This script is listed as a security exploit because, as the site states, “This allows a remote attacker fill knowledge of the router configuration, routes, etc.” And the site is absolutely correct; if your devices are accessible from the Internet, you might want to seriously consider disabling TFTP (or Simple Network Management Protocol—SNMP— which is what this script uses to force the device to send its configuration via TFTP) on them to prevent this sort of thing from happening. Fortunately, most companies’ internal routers use non-routable IP addresses (such as 192.168.0.1), meaning the routers can’t be contacted by outside attackers.

Q 6.3: We’re planning to use TACACS+ to consolidate authentication to hundreds of network devices. Is there anything that we need to be aware of?

A: TACACS+ is great for consolidating authentication. In fact, that’s pretty much why it exists. However, you need to be aware of a few vulnerabilities and some limitations.

A Brief History
TACACS (without the +) was an early military protocol, first documented in Request for Comment (RFC) 1492. Cisco adopted TACACS as a sort of favorite protocol, and released XTACACS in 1990. This new version incorporated non-RFC proprietary extensions to TACACS, making it more suitable for use with Cisco devices. In 1998, Cisco released TACACS+. Support for TACACS+ was included in Cisco IOS 11.2 and later, and allowed the tasks of Authentication, Authorization, and Accounting (AAA) to be divided among several TACACS+ servers for better scalability.

TACACS+ is a proprietary Cisco protocol, although most of its details are publicly available. TACACS+ isn’t fully backward-compatible with TACACS and XTACACS, so it’s important to distinguish which you’re using. The biggest caveat with TACACS+ is that it’s primarily supported on Cisco devices; if you assumed that TACACS+ enjoys the wide support of similar protocols such as RADIUS, be sure you check your network devices to make sure they offer TACACS+ interoperability.

>> Pretty much nobody supports TACACS and XTACACS anymore, including Cisco, so I’ll focus on TACACS+ for this tip.

How TACACS+ Works
TACACS+ is a client/server protocol. Clients, such as routers and network access devices, send AAA requests to a TACACS+ server, which is typically implemented as a UNIX daemon or a Windows background service. The server then processes and responds, if appropriate, to the request. The responding server represents the first vulnerable point in a TACACS+ implementation. First, whoever controls that service controls your enterprise-wide device authentication. On a Windows TACACS+ server, which will typically use the server’s local user accounts or a domain for authentication, whoever controls the creation and modification of those accounts controls access to your network devices. Similarly, UNIX TACACS+ servers often keep their user lists in a configuration file, and whoever controls that file controls your network devices.

TACACS+ packets use an encryption technique to encrypt client/server messages, and the encryption key is a pre-shared secret, or password, entered on both the client and server. However, TACACS+ does not implement packet integrity checking, meaning packet capture and replay is a potential vulnerability, although the encryption limits the effectiveness of these attacks.

TACACS+ also uses session numbers to track AAA communications with clients. These session numbers are nonrandom and are in fact limited to 1, 2, or 3 for authentication, authorization, and accounting. Another portion of the session ID number is determined by a cryptographic algorithm, which is also nonrandom. With a large amount of TACACS+ traffic on the network, the randomness of these session IDs decreases, potentially making it easier for an intruder to mess with packets. For example, a packet could be switched from “authorization” type to “authentication” type. However, the server would find the remainder of the packet’s structure incorrect for the new type (that structure having been based on the original type) and potentially crash, suffer a buffer overrun, or some other unexpected behavior. Note that only the body of the TACACS+ request is encrypted; the header is not, making it easy for someone with physical access to the network to capture, modify, and replay packets in an attempt to crash the server.

TACACS+ implementations before version 1.78 used User Datagram Protocol (UDP) packets to carry authentication requests, which is less reliable than the Transmission Control Protocol (TCP) connections used by newer versions. Ideally, you should configure your TACACS+ implementation to use a port other than TCP 49 (the default) to help obfuscate the service and thwart potential attackers.

Perhaps the biggest vulnerability in TACACS+ is the relative weakness of the overall encryption mechanism. It’s possible for someone with physical network access to capture an authentication request from a client and modify it. This request could be submitted to the server and accepted; the server’s reply would be encrypted, but because the clear text of that reply would be known, it would become much easier to break the encryption. Because the encryption is based on a shared secret that is rarely changed, breaking the encryption once would reveal the key necessary to break it from then on. Thus, a good practice is to regularly change the shared secrets used by TACACS+ clients to communicate with the server.

Accounting also presents replay vulnerabilities. Because the TACACS+ accounting packets aren’t sequenced, they could be captured and replayed, essentially corrupting the TACACS+ accounting log and potentially even used as a denial-of-service (DoS) attack to prevent legitimate accounting packets from being processed-essentially disguising information.

Risk Analysis
How likely are any of these attacks? Probably not very, especially if your physical network containing the TACACS+ servers and clients is reasonably secure. However, it’s always useful to know where the holes are so that you can at least watch out for exploits.

>> For more information, check out the Cisco TACACS+ document at ftp://ftpeng.cisco.com/pub/tacacs/tac-rfc.1.78.txt. There’s also a good white paper about TACACS+ flaws and vulnerabilities at http://www.securityfocus.com/bid/1294, which provides more detail than I’ve given here. You can also read the TACACS FAQ at http://www.de.easynet.net/tacacs-faq/tacacs-faq.html.

Q 6.4: How can we ensure that our devices’ software is consistent and up to date?

A: Managing your devices to ensure that they always contain the most recent version of their operating system (OS) software is becoming more important. Recent virus attacks that targeted Cisco routers directly have made it apparent that network devices—which were once largely ignored by virus authors and other hackers—are now fair game.

Interestingly, the very existence of a software patch for your routers makes them more vulnerable. The Blaster virus, which targeted Microsoft Windows-based computers, is an excellent example of how virus authors are taking advantage of lax patch management. A security firm notified Microsoft of a flaw in their remote procedure call (RPC) code that could allow an attacker to take complete control of a computer over TCP port 135. The notification was made quietly to Microsoft, which released a patch. The patch documentation did not describe in detail how the patch worked or what the original problem was. However, the patch contained only one replacement DLL file. The authors of the Blaster worm simply compared the new file to the old, then reverse-engineered the differences to determine what had been patched. They then wrote a worm to take advantage of the security hole on unpatched systems, knowing that most systems remain unpatched—often until the next service pack is released.

The Blaster worm has no direct effect on network devices, but the fact that virus authors are now using patches to develop their viruses should be a wake-up call for everyone in the IT industry: the minute a patch is released, apply it. This uncompromising approach means that you must ensure that all your network devices have a consistent, up-to-date software configuration.

Getting TFTP Ready
One way to ensure that your routers and other devices are up to date is to deploy a TFTP server on your network. This server can be used to house new updates; most devices can retrieve the update from the TFTP server. Start implementing this process by preparing the TFTP server. If you’ve got a spare Red Hat Linux server handy, you can run the following commands to make it a TFTP server:

# mkdir /tftpboot
# chmod 666 /tftpboot
edit file /etc/inetd.conf, uncomment line
tftp dgram udp wait root /usr/sbin/tcpd in.tftpd
# killall -1 inetd

Next, copy your update files—the binary files provided by your device manufacturer—into the TFTP folder and give them the appropriate permissions:

# cp update.bin /tftpboot/update.bin
# chmod 666 /tftpboot/update.bin

Preparing Your Devices
Take the time to make a backup of your devices before proceeding. You will want a backup of the entire NVRAM, especially the running configuration. After you finish upgrading the devices’ firmware, reload the current configuration to get the device back up and running.

Upgrading the Firmware
The following code provides an example of commands to use to upgrade a Cisco 2501 router to a new version of the Cisco IOS:

Username: login
Password: *******
myrouter>en
Password: *******
myrouter#sh ver

The router would then display its version banner and other information. To begin the upload process, use the following commands:

myrouter#sh flash

System flash directory:
File Length Name/status
1 5895768 c2500-is-l.112-14
[5895832 bytes used, 2492776 available, 8388608 total]
8192K bytes of processor board System flash (Read ONLY)

Next, tell the router to save its configuration via TFTP. In the following example, the TFTP server is at 192.168.1.100:

myrouter#wri n
Remote host [192.168.1.100]? 192.168.1.100
Name of configuration file to write [myrouter-confg]? myrouter-confg
Write file myrouter-confg on host 192.168.1.100? [confirm]
Building configuration...

Writing myrouter-config !! [ok]

With the configuration safely written, you can upload a new configurtion into the router:

myrouter#conf n
Host or network configuration file [host]?
Address of remote host [192.168.1.100]? 192.168.1.100
Name of configuration file [myrouter-confg]? myrouter-confg
Configure using myrouter-confg from 192.168.1.100? [confirm]
Loading myrouter-confg from 192.168.1.100 (via Ethernet1) : !
[OK — 1918/32723 bytes]

Next, write the current flash memory out to TFTP as well:

myrouter#copy flash tftp

System flash directory:
File Length Name/status
1 5895768 c2500-is-l.112-14
[5895832 bytes used, 2492776 available, 8388608 total]
Address or name of remote host [255.255.255.255]? 192.168.1.100
Source file name? c2500-is-l.112-14
Destination file name [c2500-is-l.112-14]? c2500-is-l.112-14
Verifying checksum for 'c2500-is-l.112-14' (file # 1)... OK
Copy 'c2500-is-l.112-14' from Flash to server
as 'c2500-is-l.112-14'? [yes/no]yes

The router will display “!” as it uploads the configuration to the server, then it will provide the following output:

Upload to server done
Flash copy took 00:01:18 [hh:mm:ss]
myrouter#conf t
Enter configuration commands, one per line. End with CNTL/Z.
myrouter(config)#config-register 0x2101
myrouter(config)#^Z
myrouter#sh ver

Again, the router will display its configuration information, then you’ll issue the write command to reload the router:

myrouter#wri
Building configuration...
[OK]
myrouter#reload
Proceed with reload? [confirm]

This data is what you will see as the router reloads and restarts. The router will be in boot mode, running the boot-ROM version of its OS. The following is the simplified version of the IOS and the flash memory in read/write mode:

Username: login
Password:
myrouter(boot)>en
Password:
myrouter(boot)#sh ver
Cisco Internetwork Operating System Software
IOS (tm) 3000 Bootstrap Software (IGS-BOOT-R), Version 11.0(10c), RELEASE SOFTWA
RE (fc1)
Copyright (c) 1986-1996 by cisco Systems, Inc.
Compiled Fri 27-Dec-96 17:33 by loreilly
Image text-base: 0x01010000, data-base: 0x00001000

ROM: System Bootstrap, Version 11.0(10c), SOFTWARE

The important piece of the configuration information that you’re looking for is:

myrouter uptime is 1 minute
System restarted by reload
Running default software

cisco 2500 (68030) processor (revision L) with 2048K/2048K bytes of memory.
Processor board ID 08363366, with hardware revision 00000000
X.25 software, Version 2.0, NET2, BFE and GOSIP compliant.
2 Ethernet/IEEE 802.3 interfaces.
2 Serial network interfaces.
32K bytes of non-volatile configuration memory.
8192K bytes of processor board System flash (Read/Write)

Configuration register is 0x2101

This data tells you that you’re running from the device’s internal memory, and you can start working with the flash memory:

myrouter(boot)#copy tftp flash

System flash directory:
File Length Name/status
1 5895768 c2500-is-l.112-14
[5895832 bytes used, 2492776 available, 8388608 total]
Address or name of remote host [255.255.255.255]? 192.168.1.100
Source file name? c2500-is-l.112-15.bin
Destination file name [c2500-is-l.112-15.bin]? c2500-is-l.112-15.bin
Accessing file 'c2500-is-l.112-15.bin' on 192.168.1.100...
Loading c2500-is-l.112-15.bin from 192.168.1.100 (via Ethernet1): ! [OK]

Erase flash device before writing? [confirm]
Flash contains files. Are you sure you want to erase? [confirm]

Copy 'c2500-is-l.112-15.bin' from server
as 'c2500-is-l.112-15.bin' into Flash WITH erase? [yes/no]yes
Erasing device... eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee ...erased
Loading c2500-is-l.112-15.bin from 192.168.1.100 (via Ethernet1):

The router will display “!” as it loads the configuration file. Once it is finished, it will verify the file and allow you to work with it:

[OK - 5895436/8388608 bytes]

Verifying checksum... OK (0x61A0)
Flash copy took 0:03:02 [hh:mm:ss]
myrouter(boot)#conf t
Enter configuration commands, one per line. End with CNTL/Z.
myrouter(boot)(config)#config-register 0x2102
myrouter(boot)(config)#^Z
myrouter(boot)#sh ver
Cisco Internetwork Operating System Software
IOS (tm) 3000 Bootstrap Software (IGS-BOOT-R), Version 11.0(10c), RELEASE SOFTWA
RE (fc1)
Copyright (c) 1986-1996 by cisco Systems, Inc.
Compiled Fri 27-Dec-96 17:33 by loreilly
Image text-base: 0x01010000, data-base: 0x00001000

ROM: System Bootstrap, Version 11.0(10c), SOFTWARE

myrouter uptime is 5 minutes
System restarted by reload
Running default software

cisco 2500 (68030) processor (revision L) with 2048K/2048K bytes of memory.
Processor board ID 08363366, with hardware revision 00000000
X.25 software, Version 2.0, NET2, BFE and GOSIP compliant.
2 Ethernet/IEEE 802.3 interfaces.
2 Serial network interfaces.
32K bytes of non-volatile configuration memory.
8192K bytes of processor board System flash (Read/Write)

Configuration register is 0x2101 (will be 0x2102 at next reload)

With everything in place, you can reload the router. The configuration has been changed, so the system will prompt you to save the configuration; don’t bother—simply let the router restart:




The router will come up normally; reload the configuration that the router should be using:

Username: login
Password:
myrouter>en
Password:
myrouter#sh ver

Again, the version information will be displayed. You can now load the configuration file from the TFTP server:

myrouter#conf n
Host or network configuration file [host]?
Address of remote host [255.255.255.255]? 192.168.1.100
Name of configuration file [myrouter-confg]? myrouter-confg
Configure using myrouter-confg from 192.168.1.100? [confirm]
Loading myrouter-confg from 192.168.1.100 (via Ethernet1): !
[OK - 1953/32723 bytes]

myrouter#wri
Building configuration...
[OK]
myrouter#exit

You’ve completed the software upgrade. This sequence is fairly complex and requires that you verify that each step has completed properly. In other words, this process is not easily scripted. However, software products such as AlterPoint DeviceAuthority and Cisco CiscoWorks have the ability to run through a script like this automatically, ensuring that each step completes properly. With such software, you can upload new software to multiple devices at the same time.

The benefit of using a dedicated software solution is that you don’t have to worry about individually backing up each device, running the script, and verifying that everything completes properly. The software performs these tasks for you, making it easier to manage a large number of devices. In fact, in a large enterprise with hundreds of devices, an automated software solution can be the only feasible way to keep everything updated.

Q 6.5: How can we manage device changes in real-time?

A: Device change management in real-time is a tricky process. Very few devices offer any sort of events-based notification other than standard Simple Network Management Protocol (SNMP) traps sent to a management console. In other words, routers don’t typically provide details of a configuration change; instead, they tend to notify a management console via SNMP trap that someone has logged on and done something. The notifications lack the detail that a change-management solution typical requires.

SNMP traps are a start—they at least provide some kind of notification that something has happened. To configure a Cisco IOS device (except some remote access devices) to send SNMP traps for most configuration-related operations, execute the following commands on the device:

snmp-server host 192.168.1.102 public config bgp tty
snmp-server enable traps config
snmp-server enable traps bgp
snmp-server enable traps tty

Be sure to provide the correct IP address for the SNMP workstation, which, in this example, is 192.168.1.102. Shortly, we’ll explore how this process can help your configuration-management process.

Another technique is to use TACACS+ accounting information, which is available on most Cisco devices (Cisco owns the TACACS+ protocol) and on some other devices; Remote Authentication Dial-In User Service (RADIUS) accounting can also be used, although it tends to provide less-specific information. The following configuration file enables accounting:

aaa new-model

radius-server host 192.168.1.102 auth-port 1812 acct-port 1813 key secret
radius-server retransmit 3 !
radius-server timeout 6 ! default = 5 seconds
radius-server deadtime 1 ! default = ? minutes

tacacs-server host 192.168.1.102 key secret
tacacs-server timeout 6 ! default

aaa authentication login default radius local
aaa authentication login NO_AUTHEN none
aaa authentication enable default group radius enable

aaa authorization network default group radius if-authenticated
aaa authorization exec default group radius if-authenticated
aaa authorization exec NO_AUTHEN
aaa accounting exec default start-stop tacacs+
aaa accounting commands 0 default start-stop tacacs+
aaa accounting commands 1 default start-stop tacacs+
aaa accounting commands 12 default start-stop tacacs+
aaa accounting commands 14 default start-stop tacacs+
aaa accounting commands 15 default start-stop tacacs+
aaa accounting network default start-stop tacacs+
aaa accounting connection default start-stop tacacs+
aaa accounting system default start-stop tacacs+

line con 0
login authorization NO_AUTHEN

The idea for using either TACACS+ or SNMP is that if you have a change-management solution capable of supporting the functionality, the solution can either receive SNMP traps or analyze the TACACS/RADIUS log on a periodic basis (or even directly receive TACACS/RADIUS accounting information). The traps or log information don’t provide the actual change data, but they serve as a trigger to let the solution know that someone, for example, logged off of the device. This trigger tells the solution to log on via TFTP (or whatever means for which it is enabled) and retrieve the devices’ current configuration.

By continually retrieving the configuration whenever someone logs off, you’re sure to have a complete record of changes that were made over time. This ability contrasts with solutions that simply retrieve the device configuration on a scheduled basis, because such devices can easily miss incremental changes performed between the retrieval intervals.

Thus, there isn’t really a way to perform real-time change management; however, with an automated change-management solution capable of scanning accounting logs or receiving SNMP traps, you can enable near-real-time change management by continually downloading and storing device configurations whenever anyone performs an action that might result in a configuration change. As you’re evaluating change-management solutions, be sure to consider this capability in your features comparison.

Q 6.6: What is the best way to simplify the process of reconfiguring more than one hundred network devices?

A: The ability to reconfigure multiple network devices as easily as a single device—or even more easily, if possible—is important in organizations that support dozens of network devices. Some of the most basic security practices, in particular, can be more easily implemented through an automated configuration capability.

Failing to change devices’ Simple Network Management Protocol (SNMP) community strings to something other than the default “public” is like failing to lock the doors on your car—eventually, someone’s going to steal it or something in it. However, many companies change those SNMP strings only once—when the device is first placed into service. As a result of the complexity and time required to change the SNMP strings on literally hundreds of routers, switches, and other devices, organizations simply don’t do it. However, the same organizations require users to change their passwords every 30 days because old passwords are more likely to be compromised. But old SNMP strings are just as likely to be compromised and are more likely to have a severe impact if they are compromised. Whether the changes are devices’ administrator passwords, access permissions, or routing tables across a group of devices, the ability to make consistent changes to a large number of devices is a very real need in large organizations.

The complexity of this task requires more than simply write a script or two; you need third-party software to provide the correct combination of workflow, configuration deployment, change validation, and more. Solutions from AlterPoint, TripWire, Ecora, and Voyence can provide all or part of a mass-configuration-management solution. You’ll need to carefully evaluate these companies’ offerings to determine which solutions meet your business requirements.

For example, AlterPoint’s DeviceAuthority provides an Update Module that allows you to automate the process of changing several devices’ configurations at the same time. Figure 6.1 shows the module's overview page for a password-update job. You can see that only a portion of the inventory—the list of devices that the software knows about—will be modified; the Austin Inventory selection suggests that only devices in an Austin office will be modified by this job. The job also includes a workflow, which I’ll examine in more detail in a bit. Notice that the job includes a notification capability, which will provide email notifications to selected users when the update is run, providing immediate status feedback.

>> Software vendors use different techniques for managing multiple device updates. DeviceAuthority uses a “job” to represent a single set of changes, and provides a “workflow” model to handle devices that are offline or not available when the update runs. Other software uses different terms. When comparing software packages, look for details like the ability to send email notifications, have update scripts created for you, and handle multiple devices simultaneously.

Figure 6.1: Deploying a configuration change to multiple devices at the same time.

Figure 6.1: Deploying a configuration change to multiple devices at the same time.

A useful function is the ability to specify different actions for various parts of the update process. With DeviceAuthority, you can specify a different update script for the start of the update process, before each device is updated, after each device is updated, and after the entire job finishes. As Figure 6.2 shows, a short script can be used to ping each device before trying to update it; a failure will cause that device to be skipped and the failure to be logged. After updating each device, the device configuration is backed up, ensuring that the new configuration is safely stored and that the organization's overall configuration management process is maintained.

Figure 6.2: Customizing deployment workflow.

Figure 6.2: Customizing deployment workflow.

Figure 6.3 shows another important feature—the ability to provide detailed status reports for each step of the update process. Ideally, this status report is available both from the software's management interface, as Figure 6.3 shows, and through email notifications. The email notifications help provide real-time updates and status checks to administrators throughout your organization, helping ensure that everyone is kept up-to-date with the status of the change.

Figure 6.3: Viewing update results in DeviceAuthority.

Figure 6.3: Viewing update results in DeviceAuthority.

The following list highlights key features to look for in a software package:

  • The ability to target specific devices for an update. For example, you might want to modify only devices at a certain office or only devices that are running a certain operating system (OS) version.
  • The ability to have rudimentary workflow in the update job. This workflow should allow actions that skip a device or cancel the job entirely, providing the job with some intelligence to handle real-life network conditions.
  • The ability to incorporate management workflow into the process. For example, the software might allow an administrator to create a job, then require another administrator to review and approve the job before the job can actually execute.
  • The ability to schedule jobs. You might want certain updates to run during off-hours. You should also be able to specify blackout times when no jobs are allowed to run, helping ensure that an administrator doesn't accidentally schedule a job that restarts devices to run during a period in which devices must be available. These scheduling capabilities can help you meet service level agreements (SLAs) and maintain a working production environment.
  • A small built-in library of utility scripts. These scripts perform useful work such as pinging devices to check for availability. Alternatively, they might be example scripts to help you more quickly generate the appropriate update scripts.
  • A known scripting language. Most software includes a scripting capability that allows you to perform complex tasks in an automated fashion. This scripting should use a known, documented language—such as JavaScript—or at least provide a well-documented proprietary language that includes plenty of examples. Ideally, the software monitors your actions on a device and creates a script based on those actions that can then be deployed to multiple other devices.

Selecting a solution that employs configuration templates—or developing your own templates, as I discussed in Tip 3.8—is another way to ensure that updates to multiple devices are made smoothly, consistently, and with less error. The ability to use configuration templates might be another selection criterion for a device management solution.

Q 6.7: How do you configure new devices to have the same consistent configuration as existing devices?

A: Ideally, you will create templates and scripts, which can be deployed to as many devices as possible. By eliminating manual configurations steps, you'll also eliminate error and ensure consistency.

>> Using scripts and templates also makes your change-management process and a peer review much more important. Any errors that creep into your scripts or templates will be propagated to all of your devices. Thus, it is important that someone checks and tests scripts or templates before they are used to modify multiple devices.

Creating such scripts can be a complex, time-consuming process—one that you might not find the time for in a busy environment. One trick is to find a software utility that will automatically create scripts for you. For example, Figure 6.4 shows the script-creation tool provided by AlterPoint's DeviceAuthority. This tool provides you with a Telnet session, which you use to connect to a device such as a router or switch. You can see the Telnet session in the upper half of the window. However, this tool is no ordinary Telnet client—it watches everything you type and everything that the device sends to you in response. In the lower half of the window, the tool compiles a JavaScript file that mimics your actions. The practical upshot of this feature is that you manually configure only one device, then you have a JavaScript file that you can deploy to configure multiple other devices in the same way. You can even edit the script, if desired, so that you can change device-specific information such as IP addresses.

Figure 6.4: Employ a tool that records actions and creates a script automatically.

Figure 6.4: Employ a tool that records actions and creates a script automatically.

You can also create your own configuration templates without using complicated scripts simply by storing a configuration script in a text file. For example, I once worked with a company that deployed a dozen Cisco 1720 devices to their field offices and needed the devices configured to support a T1 or fractional T1 setup with an ISDN BRI backup interface. We developed a simple, standardized configuration script (or template) for these devices, which Listing 6.2 shows.

interface Serial0
backup delay 120 120
backup interface BRI0
no ip address
no ip directed-broadcast
encapsulation frame-relay IETF
keepalive 5
frame-relay lmi-type ansi
!
interface Serial0.1 point-to-point
description company.office.serialptp
ip address 172.24.5x.1 255.255.0.0
ip broadcast-address 172.24.5x.2
no ip directed-broadcast
frame-relay interface-dlci <DLCI> IETF
!
interface BRI0
description company.office.bri
ip unnumbered FastEthernet0
no ip directed-broadcast
encapsulation ppp
dialer idle-timeout 2147483
dialer wait-for-carrier-time 10
dialer map ip 38.1.1.1 name <PAP LOGIN> <DIAL NUMBER>
dialer load-threshold 1 either
dialer-group 1
isdn switch-type <SWITCH TYPE>
isdn spid1 <SPID1> <DN1>
isdn spid2 <SPID2> <DN2>
ppp pap sent-username <PAP LOGIN> password <PAP PASSWORD>
ppp multilink
!
interface FastEthernet0
ip address 192.168.1x.1 255.255.255.0
ip broadcast-address 192.168.1x.5
no ip directed-broadcast
!
ip classless
ip route 0.0.0.0 0.0.0.0 <DEFAULT GATEWAY> 1
ip route 0.0.0.0 0.0.0.0 38.1.1.1 2
ip route 38.1.1.1 255.255.255.255 BRI0
!
access-list 101 permit ip any any
dialer-list 1 protocol ip list 101

Listing 6.2: A simple device configuration script.

>> I've boldfaced the items you'll probably need to change in this template; italicized items are the names that this particular company changed from office to office.

Creating this type of template will help ensure that nothing is forgotten when configuring new devices, and that the new devices receive a configuration with standardized IP addressing, device names, interface names, and so forth.

Q 6.8: Is there a way to quickly obtain serial numbers and other information from devices?

A: Most vendor-provided and third-party device-management software polls this information from devices and provides it for you. For example, Figure 6.5 shows a software inventory report, which you can use to see which chassis version each device is using. This information can be useful when determining which devices need updates or which devices might be affected by a particular problem that is unique to one chassis version.

Figure 6.5: Viewing devices' software versions using AlterPoint's DeviceAuthority.

Figure 6.5: Viewing devices' software versions using AlterPoint's DeviceAuthority.

If this inventory is incorporated into a change-management solution—as is the case with DeviceAuthority—the inventory is usually under change management as well. For example, suppose a company's routers were beginning to perform very poorly. Upon investigation, they realized that the routers simply didn't have enough memory installed to support the traffic they were trying to handle. The problem was that the routers had been purchased with fully populated memory slots, meaning that memory had at some point been removed. The company blamed a disgruntled, now-departed employee, but the damage was done and the memory was missing. Had the routers been under a change-managed inventory program, an administrator would have received immediate notification when the devices' hardware inventories changed, and the problem could have been caught—if not in time to save the memory, at least in time to fix the problem before it began impacting production operations.

Hardware inventory functionality, combined with change management, can also provide troubleshooting capabilities. For example, suppose a device suddenly starts to restart itself without warning. A glance at the hardware inventory report might reveal that the problem started shortly after adding a new interface card to a slot within the device. You might immediately suspect that the interface card was faulty, remove it, and have the problem fixed without a lengthy troubleshooting process.





Tips and Tricks Guide to Network Configuration Management

To download/read the eBook in its entirety, visit: http://www.alterpoint.com/ebook


myrouter(boot)#reload System configuration has been modified. Save? [yes/no]: no Proceed with reload? [confirm] Copyright 2003 Realtimepublishers.com, Inc. This text is excerpted from the eBook , Chapter 3: Network Device Management

Dig deeper on Network Performance Management

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchSDN

SearchEnterpriseWAN

SearchUnifiedCommunications

SearchMobileComputing

SearchDataCenter

SearchITChannel

Close