How to read an SLA

Many Internet and content delivery network service providers offer service-level agreements that might sound impressive – until you read the fine print.

SLA weaknesses fall into two categories: weak guarantees and weak enforcement. An example of a weak guarantee is “less than 1% packet loss.” In some situations, 1% seems quite small (for example, “1% of students got an A in the course”). In others, 1% is huge (“1% of the tires were defective”). On the Web, 1% packet loss is quite large; two lost packets for the 200 packets that a typical home page requires. Unfortunately, each lost packet tends to have a noticeable performance penalty as a result of factors such as network timeouts and slow paths through protocol code. So if your ISP delivers anything close to 1% average packet loss, look for a new ISP.

I found other examples of weak guarantees in ISP availability metrics. One vendor promised 100% availability but defined an availability incident as an hour or more of 100% packet loss. So 59-minute outages and periods of 90% packet loss aren’t availability problems for this SLA. Another SLA promised 10-minute resolution of outages but didn’t indicate if degraded service was considered an outage.

Scope limitations are a third example of a weak guarantee. Most ISPs will guarantee round-trip performance only within their networks. However, most Web user requests must traverse multiple major ISPs to reach your servers, as the largest providers host less than 50% of end users. Large sites often use multiple ISPs, providing better connections to more end users.

In fairness, intranetwork performance is the only thing ISPs directly control. Beware of promises about performance across the entire Internet – they are an example of weak enforcement. One SLA had a decent performance guarantee for domestic U.S. traffic (55-msec round-trip time) but weakened it with enforcement based on traffic from automated test clients located within the provider’s network. The result is no more of an end-to-end guarantee than the intranetwork promise.

The most common enforcement problems arise from statistical treatments applied to SLA metrics. Performance and packet- loss guarantees commonly are based on monthly averages. Note, however, that the Internet is exceptionally fast during off-peak hours. A few hours of poor performance during the day disappear when statistically blended with hours of excellent performance at night. So a service provider with bad performance could satisfy what seemed like a tough SLA. Conclude that SLAs based on monthly averages are weak.

In defense of the service providers, SLAs are weak for a good reason: The Internet is a federation. Perhaps a weak promise is better than no promise at all. My advice: Demand an SLA from your ISP, then read and understand it. Your service provider should know you understand the SLA and will be looking for a new provider long before the SLA terms are invoked.