Tip

Size matters

Contrary to small-minded beliefs, size matters -- even on networks.

Jumbo packets can make a big difference on high performance networks. Bigger packets mean bigger payloads and fewer packet headers. And that translates to faster networks -- much faster. Contrary to my last column about bandwidth, this is one area where more is definitely better.

In the history of Ethernet, IP packets have been limited in size to be a maximum of 1500 bytes. The 10 and 100 Mbps Ethernet standards define that to be largest possible packet size. Gigabit Ethernet dispensed with that limit and the standard allows arbitrarily large packets (at least up to 12,144 bytes – thereafter, CRC-32 begins to break down).

A "jumbo" packet is simply any packet larger than 1,500 bytes. There is no standard jumbo size but the industry is gravitating toward supporting sizes around 9,000 bytes (or slightly larger than 8 * 1,024). Most GigE equipment supports some form of jumbo but you have to read the fine print carefully to avoid disappointment.

So why does jumbo make a difference? Networks are rapidly outstripping CPUs. Or to be more specific, networks are outstripping NICs, drivers, and the busses they are attached to. Performance bottlenecks are increasingly found at the end-host rather than in the mid-path. Packet header generation can be costly and adversely affects throughput. Research has shown that typical workstations can't fill the pipe at GigE using 1,500 bytes, often

    Requires Free Membership to View

limited to just a few 100 Mbps. Wizards can achieve much more, for example by clever tweaking of the TCP stack. But, in many cases, simply switching to jumbo 9,000 byte packets can as much as double the achievable throughput.

Matt Mathis, a network research scientist for Internet2 and the Pittsburgh Supercomputing Center, has made the case that maximum packet size not only needs to increase to 9,000 bytes, but for reasons of future network performance needs to keep on increasing. His view is that the minimum time between packet headers needs to be kept approximately constant so that CPU and NICs don't have to work harder generating packets as the line rates increase. Larger payloads increase the separation between headers. So, for example, packet headers at 10 Mbps are separated by about 1,200 μsec -- to keep that separation at one Gbps, the packets would have to be roughly 100 KBs!

So taking a small step toward larger packets can mean immediate benefits today -- but it is only the beginning. Networks will only get faster. And the bottlenecks due to packet size will only get tighter. It's time to take size seriously.

I say, "Make mine jumbo."

More information links:
Raising the Internet MTU (Matt Mathis/Internet2)
Gigabit Ethernet Jumbo Frames (Phil Dykstra/WareOnEarth)


Chief Scientist for Apparent Networks, Loki Jorgenson, PhD, has been active in computation, physics and mathematics, scientific visualization, and simulation for over 18 years. Trained in computational physics at Queen's and McGill universities, he has published in areas as diverse as philosophy, graphics, educational technologies, statistical mechanics, logic and number theory. Also, he acts as Adjunct Professor of Mathematics at Simon Fraser University where he co-founded the Center for Experimental and Constructive Mathematics (CECM). He has headed research in numerous academic projects from high-performance computing to digital publishing, working closely with private sector partners and government. At Apparent Networks Inc., Jorgenson leads network research in high performance, wireless, VoIP and other application performance, typically through practical collaboration with academic organizations and other thought leaders such as BCnet, Texas A&M, CANARIE, and Internet2. www.apparentnetworks.com

This was first published in February 2005

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.