Problem solve Get help with specific problems with your technologies, process and projects.

Router Expert: Understanding TCP/IP to prevent network attacks, part 1

Successfully defending your depends on understanding TCP/IP operation and the potential network attacks it invites.

Read about Michael

By Michael J. Martin

Last month I gave a talk on The ABC's of IDS in a SearchNetworking Webcast. IDS (intrusion detection systems) is out of my normal topic range, but has been an area of personal and professional interest for me for some time now. So in this spirit of this topic, I thought a discussion on network exploits and some of the available IOS options for defending your Internet connection would be appropriate. But to understand that, it is helpful to know TCP/IP operation and the potential network attacks. So this month we'll look at TCP/IP, and next time we will follow up with vulnerabilities, attacks and how to defend the homeland.

In a kinder, gentler world…

The Internet Protocol suite (commonly referred to as TCP/IP) was created during the infancy of modern networking. TCP/IP was developed to provide connection-oriented and connectionless data transport services over an abstract transmission medium and unreliable transmission infrastructure. In short -- send it over anything and deliver it any way you can. The first work was started in 1973 as part of the ongoing military research in packet-based networking and the development of ARPANET (Advance Research Projects Administration Network), the prelude to the Internet. The Internet Engineering Task Force (IETF) Request for Comments (RFC) was written in 1981, with the first commercial TCP/IP implementation appearing in 1983. The suite consists of four protocols and unreliable (i.e., best effort) delivery service known as Internet Protocol (IP), defined in RFC-791. Other protocols include a network control protocol, aptly named the Internet Control Message Protocol (ICMP), detailed in RFC-792. A connection-oriented, session transport protocol called the Transmission Control Protocol (TCP) is defined in RFC-793, and a connectionless transport protocol called the User Datagram Protocol (UDP) is described in RFC-786.

At the protocol suite's inception, computers were quite large and accessed through serial terminals, and card readers were popular. Security for these systems was viewed largely as a physical access problem, not a network access one. This "limited" view of security also holds true for the operating system that made TCP/IP so popular -- Unix. Early Unix implementations provided simple password user login security and user/group access control facilities for data access, not intrusion detection and defense. These tools were developed in an open collaborative community environment to enhance the development and exchange of technology and information. With this pedigree in mind, it is easy to understand why TCP/IP has the vulnerabilities it has and why it is so alluring to some to develop new and creative ways to exploit them.

IP delivery services
IP provides connectionless, packet network (OSI-RM Layer 3/Internet-RM Layer 3) datagram delivery services for the 100-plus ULPs, or upper-layer protocols (OSI-RM Layer 4/Internet RM Layer4). Some of the more common ULPs are ICMP, protocol ID 1, TCP protocol ID 6, UDP protocol ID 6, General Routing Encapsulation (GRE) protocol ID 47, Encapsulation Security Payload (ESP-IPsec) protocol ID 50, and Open Shortest Path First (OSPF) protocol ID 89. A complete list of ULP protocols is available at

The "packet" is a combination of an IP header appended to the UPL datagram. A datagram is a transport protocol header combined with user application data. Once assembled, the IP packet is encapsulated in a network transmission frame and transmitted to the destination host over the network transport medium. The IP header is constructed out of 32-bit words; the average header is 20 bytes in length. The header contains information about the sender and destination and the data being carried. IP provides the ability to specify packet-handling options using flags in a variable length field at the end of the header. If IP options are set, additional padding may be added after the options flags to maintain the 32-bit boundary requirement while increasing the header to 24 bytes in length. In accordance to RFC 791, the maximum size of an IP packet is 65,535 octets (bytes) in length, including IP header and ULP data.

While the IP header runs a typical 20 bytes in length, ULP datagram size is variable. To reconcile these large message lengths with the physical limitations of Layer 2 transport message sizes, known as the Message Transport Unit (MTU), IP supports packet fragmentation. This allows the packet to be segmented into transmittable unit sizes that can be handled by the underlying Layer 2 transport. When packets are fragmented, the header IP fragment flag is set. When fragmentation is used, the initial packet contains the ULP information and the fragment identifier. Subsequent fragments contain only a segment of the ULP data, an offset value detailing the byte range of the fragment, and the fragment identifier. The destination host must receive all of the fragments and then reassemble the packet before deciding how to handle the IP datagram. Communication between hosts is achieved by sending packets to a unique network host identifier. IP version 4 (the current standard) uses a 32-bit address, expressed as four 8-bit numbers, with a value between 0 and 255. Values 0 (classful network address, not) and 255 (classful broadcast address) are reserved for network-wide communication. Under classless/Variable Length Subnet Mask (VLSM) addressing rules, a network (or prefix) can be any scope of addresses within the 32-bit range that is a power of 2, minus two addresses for the network and broadcast address. While the 0 and 255 are valid addresses for Class A and B network and under Classless/VLSM rules, most hosts and router IP implementations cannot handle these addresses correctly as "host" addresses; so it is good practice not to assign them to hosts.

IP is a "best effort" network delivery service. It uses a "next-hop forwarding" (NHF) paradigm through the use of intermediate-systems (IS) or "routers" to provide packet exchange between hosts connected on different IP networks. IP provides no guarantee of data integrity, sequential delivery order, or delivery at all, for that matter. Forwarding decisions are made discretely by each IS, with the forwarding determination made on the IP packet's destination address. The router performs a recursive-match lookup, comparing the packet's destination address to the network entries in its routing table. If no viable match is available, the packet is forwarded to the gateway of last resort; if no gateway is available, the packet is dropped. The advantage of the next-hop model is that it is quite resilient in a network topology like the Internet where multiple paths exist.

Available public IP address blocks and protocol ID's are assigned by the Internet Assigned Numbers Authority (IANA). IP network prefixes are assigned to ISPs and other large public networks managed by public or private organizations through regional IP address registries. In the U.S., the organization is known as ARIN (American Registry for Internet Numbers). In the E.U., it is known as RIPE NCC (RÉseaux IP EuropÉens), and in the Asia Pacific region it is called APNIC (Asia Pacific Network Information Centre).

ICMP, command and control
ICMP, like the Address Resolution Protocol (ARP) that is used to create mappings between IP addresses and network transmission protocol addresses, is a part of the Internet Protocol suite that is sort of an oddball. ARP provides a critical component of IP, but is often overlooked by network administrators as a potential connectivity problem point due to its benign function and obscurity. ICMP, on the other hand, provides IP message and control data using IP to provide transport, which sometimes leaves the ability to actually receive control messages in doubt.

Sent with a standard 20-byte IP header, all ICMP packets have the protocol ID 1 in the protocol field. The first 32 bits of every ICMP message contains three informational fields. The first eight bits indicate the ICMP message TYPE. There are 26 ICMP message types; a complete list is available at The next eight bits comprise the ICMP message type CODE, which provides additional information about the error type. The last 16 bits are the complement CHECKSUM of the entire ICMP message, for message integrity checking. There are five commonly used ICMP message types:


  • Destination Unreachable (Type 3) is sent by an IS when a packet is undeliverable. There are 15 code types associated with this message. Here are the code values for some of the more common errors: Code 0 = Network Unreachable, Code 1 = Host Unreachable, Code 6 = Destination Network Unknown, and Code 7 = Destination Host Unknown.
  • Redirect (Type 5) messages are sent to hosts to inform them that a better path exists locally. There are four code types for redirect messages: Code 0 = Redirect for Network, Code 1 = Redirect for Host, Code 2 = Redirect for Type of Service (TOS) and Network, and Code 4 = Redirect for TOS and Host. Support of ICMP redirection is not supported by all TCP/IP implementations, and in some cases only partially supported. Where alternative routes are accepted but the legacy routes are not correctly removed, inefficient routing decisions can result. To avoid this problem, it is a good idea to suppress ICMP redirects on routers. This is accomplished with the IOS interface configuration command <no ip redirects>.
  • Echo (Type 8) is used by Packet InterNet Grouper "ping" for testing connectivity between hosts. Echo messages are sent to the host.
  • Echo-Reply (Type 0) messages are sent when the host replies to echo requests.

TCP/UDP operation
Of the 100-plus ULP protocols that can be delivered by IP, TCP and UDP are the core transport protocols used by applications. TCP provides full-duplex, reliable, connection-oriented transport services used by applications that require either session (i.e., virtual terminal) or streamed-based communication (i.e., reliable data transmission). UDP provides simplex unreliable, connectionless transport services. UDP is used for non-session based applications (i.e., multicast streaming media and query/response data applications like DNS resolution). In which the data can be delivered using a single packet and the failure of delivery will not greatly impact the application.

Both TCP and UDP use an abstract communications point known as a port. A port's function is to serve as a unique communication session address, allowing the operating system to manage multiple, simultaneous communication sessions. A source or destination (communicating hosts each determine their own unique port number) port address can potentially be any 16-bit number between 0 and 65535. With this wide range of potential addresses, in order to establish a communications session a client needs to know the server port address to send its request. One approach would be for an administrator to configure servers to listen on specific ports and inform users of the service port address. This method, however, precludes the ability for unknown clients to communicate with servers. To address this, IANA has partitioned the TCP/UDP port space into three segments, with port number assignments applying to both TCP and the UDP protocol. Developers apply to IANA for a registered service port address. IANA then assigns a port address and publishes these assignments on the Internet at

The available port address space is partitioned into three ranges. The first range -- 0 through 1023 -- is known as the "well known ports." These have been assigned to "core" application services such as FTP, HTTP, and SSH. The port range 1024 through 49151 is known as "registered services." These ports are assigned custom applications that are typically not part of the standard TCP/IP suite. For example, RADIUS uses UDP port 1812 and 1813. Numbers 49152 to 66535 are for dynamic or private ports, which are allocated for client-to-server communication. However, hosts are free to choose any port over 1023 as a communication session binding point to establish a session with a known service.

Operationally, UDP is quite simple. UDP has a simple four-field header that contains source port, destination port, packet length and an optional checksum field. The UDP header is appended to the application data, encapsulated in an IP packet and forwarded on to the destination.

TCP -- in comparison to UDP -- has quite a lot of overhead, all of which is required to provide transmission efficiency and reliable delivery service. Where UDP is a simplex delivery service, TCP provides full-duplex virtual connections. Abstractly, TCP connections appear as hard connection channels supporting bi-directional data streams. Operationally, TCP segments the application data in packets and transmits them as batches, waits for delivery acknowledgement and sends more. When there is no data to transmit, the connection state remains intact but no data is sent.

Three-way handshake
TCP uses a three-way handshake to establish host-to-host communication. Like UDP, the first two header fields contain source and destination port. To open a session, a client determines a local source port and an Initial Sequence Number (ISN). The ISN is a randomly determined integer between 0 and 4,294,967,295. Communicating hosts exchange ISNs during connection initialization. Each host sets two counters: sequence and acknowledgement. In the context of a single TCP packet, the sequence number (third header field) is set by the sending host, and the acknowledgement number (fourth header field) is set by the receiving host.

The first packet contains the local source port, the server destination port, and the client's ISN. The packet's SYN (synchronize) flag is also set.

 Step 1. Host A (port 44567) >>>>>>>>>>>>[ SYN]>>>>>>>>>>>>>>>Host B (port 22) 

TCP supports in-band management communication through the use of "state" flags. The flags appear ten bits after the acknowledgement number field. There are six flags:


  • Urgent: Indicates the data field contains application control data, i.e., escape messages.
  • Acknowledgement: Indicates the data has been received successfully. The response contains the received packet's sequence number, incremented by one in the acknowledgement field. Every packet should have its "ACK" flag set.
  • Push: Indicates the packet should be moved to the head of the forwarding queue and transmitted as soon as possible.
  • Reset: The connection has been terminated and the receiving host should release all resources pertaining to this session.
  • Synchronize: Only set when establishing a TCP connection. It informs the application of the connection request. Upon receiving a SYN request, the operating system reserves resources to satisfy the connection requirements.
  • Finish: Indicates that the sending host will not transmit any more data, but will still accept data.

When the server receives the request on its known service port, it responds to the request, using the destination host's port address in the destination port header field. It allocates its own local port, contained in the source port header field. Then it generates its own ISN contained in the sequence number header field, setting the SYN flag.

 Step 2. Host A (port 44567) <<<<<<<<<<<[SYN/ACK]<<<<<<<<<<<<< Host B (port 56789) 

The client receives the server's SYN/ACK and responds with the server's ISN+1in the acknowledgement header field, setting the ACK flag.

 Step 3. Host A (port 44567) >>>>>>>>>>>>>[ACK]>>>>>>>>>>>>>>> Host B (port 22) 

Once the connection is established, data reliability is ensured using time-based, positive acknowledgement. To utilize the available bandwidth, TCP employs a dynamic flow control method known as Sliding-Window. Instead of sending one packet and waiting for an acknowledgement (like X.25), TCP streams the data, sending multiple packets at one time. It then waits for acknowledgements and adjusts is transmission flow to accommodate the path speed and capability of the remote end. The idea is to keep the packet flow rate where the available bandwidth is utilized without losing data. With every packet sent, a timer is set. If a host fails to receive its ACKs before the timer expires, the packet is retransmitted. In the case where multiple packets are lost, the host will reduce its flow rate until equilibrium is reestablished.

Four-way handshake
There are two ways to end a TCP session: using the FIN flag or using the RST flag. An RST packet is sent in response to a network outage or when a connection needs to be terminated quickly. The proper use of the RST flag allows the flag to be set by itself or in combination with an ACK flag.

A "normal" session termination uses a four-way handshake. Either host may initiate a session termination by sending an ACK/FIN packet. The reason this process is four-way rather then three-way is because TCP sessions are full-duplex, so both directions must be torn down. Here is the process walkthrough:

 Step 1. Host A (port 44567) >>>>>>>>>>>>>[ACK/FIN]>>>>>>>>>>>>>>Host B (port 22)

Step 2. Host A (port 44567) <<<<<<<<<<<[ACK]<<<<<<<<<<<<<<<< Host B (port 56789)

Step 3. Host A (port 44567) <<<<<<<<<<<[ACK/FIN]<<<<<<<<<<<< Host B (port 56789)

Step 4. Host A (port 44567) >>>>>>>>>>>>>[ACK]>>>>>>>>>>>>>> Host B (port 22) 

Only a malformed packet will appear with only a FIN flag set; an ACK flag is always set along with a FIN. Another use for the FIN flag is in TCP service scanning when the query host sends FIN packets at service ports on the target host. If the port is open, no response will be made. If the port is closed, the host will respond with an RST packet. Ports that fail to respond are open; ports that send an RST are closed. The advantage to FIN scanning over SYN scanning is that it is undetectable. However, certain TCP/IP implementations respond to all FIN packets with RST packets, making this method unreliable in some cases.

Flows and state
A flow is a relatively new abstraction used to describe any Internet connection. It is comprised of four identifying numbers -- source IP address: source port/destination IP address: destination port. These four values are all of the required addressing information needed for the data to be delivered. Packet-switching routers use flow abstracts for making forwarding decisions. A host-to-host flow is identified, a forwarding decision is made and the packet is sent on. The flow and the forwarding data are then cached. When the switch sees more packets with the same flow signature, they are forwarded using the cached lookup data. This reduces the time it takes to forward the packet, resulting in better performance. Flow tracking is also required with Network Address Translation (NAT) and Port Address Translation (PAT). In the case of NAT and PAT operations, flow tables are needed for tracking internal to external communication sessions.

Another use of flows is in traffic monitoring and system profiling for utilization statistics and intrusion detection. If you are looking for bandwidth management and profiling, Cisco's Net Flow switching and Net Flow Collector software allow you to collect flow data right from your routers. On a tight budget? You can use free Net Flow collectors and visualization tools like CAIDA's Cflowd and FlowScan. In the IDS space, the Ohio State University has published research on how to use Net Flow as a Network Intrusion Detection System (NIDS). On the commercial side, Lancope has just released version 2.0 of its standards-based Stealth Watch flow collector/IDS console.

The same data points are used by dynamic filtering firewalls (DFF) such as products from Cisco, NetScreen and Check Point. With DFFs, traffic is filtered based on the information in the header through a traffic filter rule base. Permitted traffic is forwarded and denied traffic is dropped in much the same way a Static Access Control List (SACL) filters traffic.

The operational difference between DFF and SACL is that with DFFs the origin state is all access denied, while with SACL the permitted service ports are always "open," leaving access points available to be exploited. Upon receiving acceptable traffic, DFFs make a dynamic permit entry and allow the session to complete. While both DFF's and SACL work at the network level, DFFs have the ability to inspect the packet up to the OSI-RM application layer. This allows filtering decisions to be made on transport layer control flags and data content if desired. DFFs are commonly referred to as "stateful" firewalls. This reference is based on how DFFs manage Dynamic Access Control List (DACL) entries.

DFFs track both the state (half-open SYN, open ACK, and closed ACK-FIN) and context (source IP: source port/destination IP: destination port) of a session as it traverses the firewall's interfaces. The data is then recorded in a "state table" that resides in the firewall's DRAM. This record is used to retract the DACL entries once a session is complete. But it also allows the DFF to use historical context to base further access requests. What DFF's cannot do is protect against attacks that reside within the ULP payload, such as buffer overflow attacks and pathological fragmentation attack. DFFs also cannot defend against attacks that are "wrapped" within normally accepted traffic like SSL or HTTP.

I hope this overview of protocol operation and vulnerabilities is helpful to those of you getting started with intrusion detection. Next month we will look at the different types of vulnerabilities and attacks that you need to learn to defend against.

Was this article helpful to you? Do you have suggestions for topics you'd like to see covered? Please  e-mail us and let us know!

This was last published in July 2002

Dig Deeper on Network protocols and standards