Network World has teamed with Visual Networks to bring you the first in a series of quarterly reports on the top ISPs in the market.
Through statistical analysis of Visual's Internet BenchMark data (see How we did it), we ranked the top ISPs in the business-to-business, national and regional ISP markets. In addition, we've tracked results since August 1999 to show whether an ISP's ranking has spiked for one month or is consistent.
In December 1999, AT&T WorldNet came out on top, performing above average in six out of nine categories tested. Coming in a close second is BellSouth, which performed above average in five out of nine categories.
| ISP | Score* | Strengths |
| AT&T WorldNet | 18 | Low CFR %** (all three categories), initial modem connect speed, average time to log on, low Web fail/timeout % |
| BellSouth | 17 | Low CFR % (evening), initial modem connect speed, average DNS lookup time, average Web throughput, average download time |
| Bell Atlantic - North | 15 | Initial modem connect speed, average DNS lookup time, low Web fail/timeout % |
| UUNET (GridNet) | 15 | Initial modem connect speed, average time to log on, average DNS lookup time |
| MindSpring | 14 | Initial modem connect speed, average time to log on |
| Ameritech | 13 | Average time to log on |
| AT&T GNS | 13 | Initial modem connect speed |
| Concentric | 13 | Average download time |
| EarthLink | 13 | Low Web fail/timeout % |
Month-to-month comparison
| ISP | Aug. | Sept. | Oct. | Nov | Dec. |
| AT&T WorldNet | 14 | 15 | 13 | 14 | 18 |
| BellSouth | 12 | 13 | 13 | 14 | 17 |
| Bell Atlantic - North | 19 | 12 | 14 | 13 | 15 |
| UUNET (GridNet) | 12 | 13 | 12 | 13 | 15 |
| MindSpring | 10 | 13 | 14 | 14 | 14 |
| Ameritech | 13 | 14 | 13 | 13 | 13 |
| AT&T GNS | 14 | 13 | 18 | 14 | 13 |
| Concentric | 12 | 15 | 10 | 15 | 13 |
Industry averages for December, 1999
| Category | National retail | Regional retail | Business to business | 24-hour CFR% | 4.8 | 3.4 | 4.3 |
| Evening-hour CFR % | 7.6 | 5.0 | 5.4 |
| Business-hour CFR % | 5.5 | 4.4 | 5.3 |
| Initial modem connect speed | 46.85 | 45.17 | 46.63 |
| Average time to logon (seconds) | 29.3 | 32.49 | 31.76 |
| Average DNS lookup time (msec) | 533.42 | 459.47 | 454.06 |
| Average Web throughput (Kbps) | 4.1 | 4.19 | 4.27 |
| Average download time (seconds) | 23.01 | 25.16 | 24.91 |
| Average total Web fail/timeout % | 3 | 2.5 | 3 |
Top ISPs by category
Business-to-Business
ISP Score UUNET (GridNet) 15 AT&T GNS 13 Concentric 13 ICG-NetAhead 12 PSINet 12National Retail
ISP Score AT&T WorldNet 18 MindSpring 14 EarthLink 13 Juno 12 MSN 12Regional Retail
ISP Score BellSouth 17 BellAtlantic - North 15 Ameritech 13 Pacific Bell 12 SBIS 11 US West 11
The data for this report comes from Visual Networks. Through its Internet BenchMark data, Visual rates national, regional and business-to-business ISPs according to several factors, including call failure rate, modem connect speed and Web download performance.
We took the raw data from Visual and applied statistical analysis to rate the relative performance of each ISP. First, we derived the standard deviation of the numbers in each performance category. Standard deviation is a measure of how far the numbers in a series diverge from each other. For each category in which an ISP performed better than one standard deviation from the industry mean, we awarded it one point. If the ISP did better than two standard deviations from the mean, we awarded it two points.
Similarly, if an ISP did worse than the industry mean by more than one standard deviation, we took a point away from its score. If it did significantly worse, meaning two or more standard deviations, we took away two points. We started with a baseline of 12 points, so if an ISP scored 0 points it ended up with an adjusted score of 12.
After scoring each category, we summed the results to produce a single number that indicates the reliability and performance of each ISP. A rating of 12 means an ISP was about even with its peers, or the industry average. In December 1999, 10 ISPs scored higher than 12, making them eligible for our list of the best national ISPs. The bad news, however, is that 11 ISPs scored less than 12 points.
Quick tip
Visual's Internet BenchMark provides a standard way to compare the overall performance of dial-based ISPs. But the performance experienced by specific users may vary based on many factors, including the type of equipment, dialing locations and the level of service purchased from the ISP.
Many companies are turning to service-level agreements (SLA) to ensure that their users are receiving the quality of service promised by the ISP. But how can users measure the day-to-day service-level metrics, such as availability or response time, negotiated from their ISPs?
Systems and tools that let enterprises and ISPs measure service levels, and manage IP connectivity and applications are typically referred to as service-level management (SLM) systems. These systems offer tool sets that define, monitor, and administer SLAs.
Data collected from an end user's point of view can be presented in a collaborative fashion, in which ISPs use SLM systems to monitor performance on dial-up and dedicated Internet circuits, and then share the performance data with their customers.
ISPs tested
| ISP | Type |
| Ameritech | Regional Retail |
| AOL | National Retail |
| AT&T Worldnet | National Retail |
| AT&T (GNS) | Business-to-Business (B2B) |
| BellAtlantic | Regional Retail |
| BellAtlantic-North | Regional Retail |
| BellSouth | Regional Retail |
| CompuServe | National Retail |
| Concentric | B2B |
| EarthLink | National Retail |
| GTE.net | National Retail |
| GTEInternet | B2B |
| ICG-NetAhead | B2B |
| Juno | National Retail |
| MindSpring | National Retail |
| MSN | National Retail |
| PacBell | Regional Retail |
| Prodigy | National Retail |
| PSI | B2B |
| RCN | Regional Retail |
| SBIS | Regional Retail |
| Splitrock | B2B |
| USWest | Regional Retail |
| UUNET | B2B |
| UUNET (GridNet) | B2B |
