This research paper "Using TCP/IP Traffic Shaping to Achieve iSCSI Service Predictability" is one of a series of papers presented at the Large Installation System Administration (LISA) 2010 conference in San Jose Nov 7-12. Other LISA 2010 papers covered a range of topics from creating firewall policy fault models to using Netflow-based analysis tools.
An increasing number of enterprises are turning to centralized and consolidated storage to better support server virtualization and private cloud environments. Specifically they are implementing SANs based on iSCSI that generally lack performance predictability. This results in the inability to ensure SAN QoS at a time when it's more necessary than ever to ensure application performance. So, what if networking and storage teams could use traditional TCP/IP traffic shaping techniques to better manage and ensure iSCSI SAN traffic?
At LISA 2010 this week in San Jose, researchers J. Bjorgeengen of University of Oslo and H. Haugerud of Oslo University College presented a paper that proposes doing just that. The paper, entitled "Using TCP/IP Traffic Shaping to Achieve iSCSI Service Predictability" introduces methods for throttling traffic to iSCSI target servers to enable traffic prioritization.
Researchers start from the concept that consolidated storage systems in which virtual disks rely on a shared storage pool of underlying hardware are less dependable than dedicated local storage units. Therefore, prioritization of workloads is necessary to control resource utilization and ensure SAN QoS. The way to do this is to implement throttling for workload prioritization that can be dynamically changed depending on need. Researchers suggest a method that introduces throttling on both read and write requests, with decision making being orchestrated by a Proportional Integral Derivative (PID) controller.
While throttling outgoing -- or read requests -- is somewhat simple, throttling write requests is more difficult. The most common form of shaping for inbound traffic is ingress policing, which drops packets from a sender when a bandwidth threshold is crossed. Using this method, the congestion control mechanisms then adjusts the sender rate to a level that can be maintained without dropping packets. But this isn't a great method for SAN QoS since packet loss could lead to inefficient network link utilization due to packet retransmits, and there may be too much time involved in the sender adapting when the receiver decides to change allowed bandwidth.
Instead, the researchers propose a different method of throttling designed to address these limitations.
"The method implies introducing a variable additional delay to packets sent back to initiators, the clients in SCSI terminology. Read requests are simply throttled by delaying all outbound packets containing payload. Outbound ACK packets containing no payload are delayed in order to throttle write request without dropping packets. The actual delay is obtained using the netem module of Linux Traffic Control, and packets are assigned different delays based on Iptables marks," the researchers explain in the paper.
Deciding when to apply throttling
Throttling decisions must be made based upon capacity availability. But the state of available capacity doesn't remain constant since it depends on rate, direction and pattern of workloads. Researchers suggest determining resource saturation by measuring consumer response times. "An Exponentially Weighted Moving Average (EWMA) of the response time is applied before it is used as the input signal to the control mechanism," the researchers write.
To read the specifics of how to apply traffic shaping measures to iSCSI storage environments, read "Using TCP/IP Traffic Shaping to Achieve iSCSI Service Predictability" in its entirety. Also, view a complete listing of the research papers presented at LISA 2010.
This was first published in November 2010