What can you tell me about core I/O networking cards and 10GbE?
I have several questions:
What are core I/O networking cards used for in a customer's datacenter?
Do you see 10GbE acting as a core I/O, especially on high end UNIX servers, mainframes and others?
In a previous article you mentioned that 10GbE BASE-T could be a replacement card for all the different flavors of GbE. Could you please elaborate on that?
How would you position InfiniBand with 10GbE? How and where do you see the implementation of these two technologies at the customer's datacenter?
First, as to core I/O cards, the answer is it really varies a lot, but the trend is for gigabit and 10G. There is a new standard for two pair gigabit that is getting quite a bit of press. It is written for Category 7/Class F, but is part of the fiber channel standard. The cool thing here is that you could get two 1G connections out of a single RJ45 footprint. Data center space is very expensive so anything that is standards-based and will provide great ROI over revisiting any infrastructure is key. The benefit of copper over fiber is two fold. One, copper is less expensive and the active electronics are less expensive. The second is that you can provide power over copper which may save on electrical connections. Third, but less important, is that copper will auto-negotiate (fiber will not), so an investment in a core switch can be preserved as speed needs increase. While fiber will support greater distances, this is often not a factor in a data center for many of the connections.
10GbE is certainly huge. In answer to your next two questions, 10GBASE-T auto-negotiates between 10/100/1000/10G. So if a company was looking at longevity in their investments, this could be a huge savings even if they are an early adopter when prices are higher. They could use 1G speeds and progress up to the higher speeds by simply changing out a NIC card as they needed the speed. This means that a switch would last longer and provide a greater lifespan than say a 1G switch that would eventually have to be replaced. The cost of making a chip is cut in half every 18 months so the new 10GBASE-T chips will eventually become very reasonable.
As for InfiniBand or 10GBASE-CX4, the main problem with this technology is the distance limitation of 15m and the fact that it runs on coax instead of a standard structured cabling system. In many data centers, 15m is not sufficient to go from one row of racks to another which may mean that some rearrangement will be necessary (this is VERY expensive in a data center due to the number of users that rely on the connections). I personally believe that 10GBASE-T will replace InfiniBand – it would be relatively simple for InfiniBand manufacturers to update to 10GBASE-T – but that remains to be seen. I have been told that at least a few have pulled the products in anticipation of releasing 10GBASE-T components instead. Please don't ask me to name names.
This was first published in August 2005