Rawpixel - Fotolia
February might fuel images of love and roses, but the month also means something much less romantic and much more prosaic: taxes. And for Theodore Turner Sr., a network engineer at tax preparation software developer Intuit, February -- not April -- can be the cruelest month.
That's because February is when millions of Americans -- with W-2s fresh in hand and visions of refunds dancing in their heads -- rush to file their taxes. Traffic to Inuit's 17,000 Web and mobile servers balloons during the first few weeks of February -- as it does in the weeks right before the April 15 deadline.
System availability, always a critical issue no matter what time of the year, becomes an even more acute concern at these high-traffic times. Downtime is not an option and neither are sluggish applications; Intuit wants all of its Web pages displayed in two seconds or less.
To ensure that Intuit is meeting customer expectations, the company a few years ago created a Performance Analysis Services team to enable the company to keep closer oversight of its network infrastructure.
Pinpointing the culprit in network, application performance problems
"We created the team that included operations, network engineers and application development," Turner said. "We wanted to be able to troubleshoot from a network perspective. Intuit is constantly evolving, and as we were bringing online new services and features, there is always troubleshooting that needs to occur, and we needed to understand whether performance problems were network-related or application-related or something in between."
To do that, Intuit tapped Riverbed Technology, expanding an existing Riverbed network monitoring deployment to a full suite of SteelCentral application performance monitoring appliances and software.
Theodore Turner Sr.Intuit
"We wanted to look at the communications between the Web and the app, and not just [the] Web server, and that's when we expanded the Riverbed visibility, so we could see the different tiers. That's when things went from a networking perspective to a performance analysis perspective as we got into the different tiers," Turner said.
Intuit uses the Riverbed platform -- in conjunction with dozens of other network analysis tools -- for both its consumer-facing and internal apps, Turner said. The company, although best known for its TurboTax, QuickBooks and Quicken product lines, has 170 business units and 800 applications.
Diagnostic capabilities boost visibility, ability to scale
"This increases our ability to bring new services online and to scale them properly," Turner said about the organization's performance analysis capabilities. "Teams aren't just relying on programmatic troubleshooting or logging and reviewing the logs. We can get insights right then and there and get feedback when [engineers] aren't seeing what they expect from the application tier."
The ability to quickly diagnose problems yielded dividends after the launch of one new service, Turner said. "We set up a load test and saw that even though requests were hitting the front end, we couldn't see any transactions on the application tier; no data was being written to the database and we knew something should have been there."
The performance team, using the Riverbed analysis platform, was able to determine the problem and resolve it the next day. "Things like that would have taken days, if not weeks, to fix" without analysis tools, Turner said.
War room set up to quickly react to tax season issues
Intuit reacts to system problems as quickly as it can, but during the February and April crunch times, the vendor convenes a war room to ensure an even faster response to performance issues. The war room is staffed 16 hours a day by Intuit engineers and representatives from vendors whose products support Intuit's software.
Intuit's engineers troubleshoot from the war room, identifying any root cause of an issue within minutes, Turner said, adding that troubleshooting time overall has declined by 80% since using SteelCentral.
Intuit's beefed-up performance analysis capabilities, meanwhile, have enabled its engineering team to play a much more active role in plotting the company's overall strategic initiatives. "I'm attached to the tax team right now, so that's where I'm providing insights so that executives can see what's going on [in the development of new services], Turner said.
Next up are plans to anchor performance monitoring to mobile and software as a service initiatives, as well as to tie monitoring closer to the components that Intuit's applications comprise. "Whether it's Java or .Net, we want to create a better correlation so that if there are discrepancies -- say with a new Java patch or a POODLE bug -- we know what's going on to determine if we want to go back to redo the code or scale up the hardware," Turner said.
Intuit also wants to fortify its baselining capabilities "to better understand how things are working well and what things look like when they aren't," Turner said. "We are always striving to shoot for 110% or 120% or 130% of the traffic we are getting, and we want to be able to take on that traffic. We have to understand baselines, so we can use that information not just for baselining, but to understand the targets of opportunities."