Most servers these days come standard with at least two, and sometimes three, Ethernet interfaces. While you may only "need" one of them, this tip will give you some ideas about what to do with the remainder.
The first use that comes to mind is to build a separate network. This could be an "out-of-band" network for administrative purposes, or it could lead to a dedicated switch for tape backups or a Storage Area Network (SAN).
Another common idea is to build a very cost-effective Linux-based firewall using IPchains or IPtables, etc. This may not be the most secure firewall ever built, but in terms of "bang for the buck" it's very hard to beat, because of the low cost of adding interfaces to support different zones.
Speaking of zones, that leads us to an important caveat. It is rarely a good idea for servers to have interfaces on more than one zone. For instance, you wouldn't want your e-mail server to have one interface on your inside network and another on the open Internet. It's much better to have traffic pass through a dedicated, stateful firewall. Don't rely on token "hardening" or adding desktop firewall software either, as your servers are more often compromised through the open ports of the applications they host; in this example, SMTP for e-mail.
Yet, that doesn't mean you can't have a SAN or out-of-band management network. It just means you should be careful with what other devices you put on those networks. Make sure the path of least resistance goes through your firewall. And I recommend, in most cases, considering things like dedicated backup subnets part of the same zone as the one the servers' primary interfaces are in.
So what other uses are there for unused NICs? Well, depending on what your hardware and drivers support, you can bundle two NICs together to form a single logical pipe with twice the bandwidth. This is very easy to do automatically if your drivers support Port Aggregation Protocol (PAgP) and Cisco's Etherchannel. Multi-link Trunking (MLT) is another option.
And of course, there's always redundancy. There are several different ways to configure multiple NICs to provide failover support, from MAC-layer solutions to DNS round-robin and lots of stuff in between.
About the author:
Tom Lancaster, CCIE# 8829 CNX# 1105, is a consultant with 15 years experience in the networking industry, and co-author of several books on networking, most recently, CCSPTM: Secure PIX and Secure VPN Study Guide published by Sybex.