Performance and raw costs for distribution capacity are two top factors to consider when a company evaluates content distribution networks (CDN) and associated services for accelerating their cloud applications and services. These decision points are muddied by many CDN vendors saying their products give the best performance. Often, they also have tests to back up their claims. Separating hype from fact is even more difficult when companies are evaluating ease of management, long-term commitments, the possibility of lock-in and support for value-added services.
Any vendor that claims its performance approach is the only viable one bends the truth, according to Dan Rayburn, principal analyst for digital media at Frost & Sullivan. The most important starting place is to look at how Web applications can improve business value using techniques like content distribution, dynamic acceleration and protection from cyberattacks. The next step would be to create a testing infrastructure for assessing performance changes, then correlating these with business value.
Any vendor that claims its performance approach is the only viable one bends the truth.
For example, WallEyeDirect.com, a specialty fish distributor, was seeing a long-term conversion rate of .85%, said founder Jesse Ness. Within a month of setting up the Yottaa Engagement Cloud CDN and dynamic site acceleration service, WallEyeDirect's conversion rate jumped to 1.25%. This boost in sales allowed the Yottaa service to pay for itself almost immediately.
For Ness, the service's key attraction was its automation and integration of various site acceleration services. "The advantage was that it was more of an automated thing. I just pay a monthly fee to get the work done automatically. There are a lot of things you can do to make your site faster, but doing it manually is not easy," he said.
Consider the big picture
CDN vendors are starting to focus on value-added services that promise to deliver the best performance with the least amount of enterprise IT resources. The various approaches to content distribution networks come with various benefits and tradeoffs. It's a good idea for a company to start with a strong idea of what it intends to optimize. These can include:
- Reducing the time spent waiting for content
- Reducing the cost of delivering content by offloading work to specialized content distribution servers
- Improving a website's resiliency using a CDN specialized in detecting and mitigating the effects of large scale cyberattacks
Rather than starting with the idea of a particular CDN capability, Forrester Research recommends starting with the goal of scaling cloud services to identify the best fit for an organization. Companies need to look at the costs and tradeoffs of using tools to add more hardware, storage and network capacity, and compare them to optimizing the existing architecture with services like content distribution and dynamic site acceleration, and services for specialized content, such as video. Most enterprises favor expanding capacity over optimizing resources, which could be a missed opportunity.
If basic service cost is the main concern, a commodity provider like MaxCDN might be a good fit. But as other factors begin to dominate, other CDN providers might be a better fit for a company's needs, said Jing Lee, systems administrator at Rent the Runway.
For those who are new to CDN, Lee recommends getting familiar with the technology using a zero lock-in, no-commitment, low-cost CDN. Then companies can gauge the traffic and return on investment after a few months. Following that, it will be easier to understand how other CDN services with more extensive ecosystems might be a better fit.
Estimate the numbers
All of the leading cloud providers offer basic calculators for estimating the cost of a CDN rollout under different load conditions. These include Amazon CloudFront, Microsoft Azure CDN and Azure Media Services, and Rackspace Cloud Files. Smaller CDN providers with value-oriented CDN offerings like CDN77 and MaxCDN provide similar calculators. However, most of the larger CDN vendors differentiate themselves more on value-added services and therefore don't offer basic cost estimation tools or pricing online.
For a basic point of comparison, here are the prices different networks charge to distribute 10 terabytes of traffic in the U.S. and Europe:
- CDN77: $490
- MaxCDN: $499
- Rackspace Cloud Files: $1200
- Amazon CloudFront: $1200
- Verizon EdgeCast: $2,130
- Akamai: $3,500
While most CDN providers offer some form of dynamic site acceleration, some CDNs are easier to deploy, depending on the use case. Brian Altenhofel, sysadmin at the Oklahoma Shooters Association found that both Rackspace Cloud Files and Amazon CloudFront did a good job of accelerating his basic content. That said, CloudFront proved better at managing the distribution of user-generated content, such as custom avatars, profile pictures and attachments.
Start with the test
In order to get an accurate assessment on the value a CDN service would bring to a business, it is a good idea to start with testing, Rayburn said. If a company plans to optimize existing applications and content, performing a thorough analysis of performance can provide a baseline for determining how well a particular CDN or service is working. For new applications, good testing can make it easier to evaluate the performance of different CDN approaches.
There are many ways to measure the performance of a website. Some include time to first byte (TTFB), page load time, up-time, caching capabilities and video quality. There are two basic techniques for assessing the performance of web applications to end users. Independent testing companies like Compuware APM can perform quick distributed testing through a cloud-based testing service.
The other approach is to instrument the client application to collect live performance data using tools like Akamai Mobitest. Podjarny argues that real user tests have the most accuracy because it reflects the kinds of latency and delays found in the real world. He explained, "In pretty much all cases, the real user measurement numbers are 2-4 times higher than synthetic measurements. Usually synthetic measures are biased towards clean, fault free, and usually faster environment. That is sadly not the real world, especially in mobile."
Assessing the performance gains of video distribution is a bit trickier since network delays and bottlenecks can impact the user experience in different ways. Rayburn said Cedexis has become the gold standard for comparing video quality over time and across CDN providers.
A good testing infrastructure can also make it easier to determine when CDN providers fail to meet their SLAs. Frost & Sullivan's Rayburn said that most of the service-level agreements used in the industry are proactive rather than reactive. If there is an outage and no one notices it, the provider will not offer a refund. Companies need to notice outages and proactively call their provider in order to receive a credit.
Website performance is important, but it needs to be interpreted in the context of business value. Forrester Research noted that a recent survey found that only 20% of firms reported measuring the revenue impact of page load delays.
This suggests that Web performance metrics need to be dramatically improved at most firms if they hope to keep pace with the importance of their online channels. A good practice is to compare the impact of performance improvement on purchases, cost of sales and sentiment about the brand on social networks. This kind of analysis is vital for creating a real business case for content distribution networks and associated services.
Important questions to ask when you evaluate content distribution networks
What kind of content do you need to support?
Static content like large video files and dynamic content like modern Web applications leverage different optimization techniques, management challenges and cost structures. Starting with the end goal in mind makes it easier to choose the best CDN platform or platforms for your needs.
What are the service costs for the main use cases?
CDN costs can vary widely depending the level of quality desired and the location of users. Expect to pay more for content distributed to South America and Africa compared to Europe and North America.
How well does the CDN perform for the use cases your enterprise needs?
All major CDN providers claim to offer the best performance, which is likely true, given the right kind of test. However, the performance and costs of the various CDN services can vary by use pattern and distribution requirements. The best way to determine which service is right for an application is by testing its implementation across the CDN providers on your short list.
It might be that the best option ends up being a combination of CDN providers. For example, San Francisco-based testing company Optimizely found it got the best performance by leveraging two CDN providers. But this could introduce new management headaches and could reduce the volume of traffic and discounts available from each CDN provider.
How well does the CDN mesh with the existing application infrastructure?
With so many CDN providers to choose from, it may be the case that a particular provider provides better integration with the existing application platform. For example, an application built on top of Amazon Web Services' Elastic Compute Cloud might be able to better leverage the Amazon backbone to allow better performance and management using CloudFront rather than a third-party provider.
What management challenges and costs are involved in leveraging a particular CDN capability?
Static and dynamic CDNs come with different management challenges. With relatively static content, the main challenge lies in reliably updating the caches of data distributed across the Internet. The deployment team in the company needs a process in place to ensure these distributed caches can be refreshed as new content becomes available.
Read part 1 of this CDN series.