WAN Services /

Top ISPs Report

November 2000 report

Here's our latest quarterly report from Network World and Visual Networks as we look at the top ISPs in the market. Through statistical analysis of Visual's Internet BenchMark data for dial-up service (see How We Did It), we ranked the top ISPs in the business-to-business, national and regional ISPs, as they compare to other vendors within the same market. We've also included results for the past six months, to show whether an ISPs ranking this month has spiked or is consistently above average.

In November 2000, AT&T's WorldNet service blew away the field in the national retail market, performing above the industry average in eight out of nine categories tested. The regional retail market continues to bunch together, with BellSouth, SBIS, Verizon North and Verizon South all performing slightly above average. In the business to business space, AT&T's Global Network Service (GNS), NaviPath and UUNET shared the top spot.

Next month, watch for a new look to the ISP report in Network World.

November 2000's top ISPs, by category
Month-to-month comparison
Industry averages for November 2000
How we did it
Quick tips
Who we tested
Top ISP archive

November 2000 Top ISPs, by Category
ISP	Score	Strengths
National Retail
AT&T WorldNet	20	Low CFR% (all three areas), initial modem connect speed, average time to login, average DNS lookup time, average Web throughput, average download time
Regional Retail
BellSouth	13	Initial modem connect speed
SBIS	13	Low total Web fail/timeout %
Verizon-North	13	Average time to login
Verizon-South	13	Low total Web fail/timeout %
Business-to-Business
AT&T (GNS)	13	Average time to login
NaviPath	13	Average Web throughput
UUNET	13	Average Web throughput

(back to top)

Month-to-month comparison for past six months

ISP June July August Sept. October November

AT&T WorldNet 20 20 16 15 15 20

AT&T (GNS) 15 15 14 13 14 13

BellSouth 13 14 12 14 13 13

NaviPath 10 12 12 12 12 13

SBIS 12 11 11 14 13 13

Verizon-North 12 10 11 10 14 13

Verizon-South 11 11 9 11 10 13

(back to top)

Industry Averages for November 2000

Category National retail Regional Retail B2B Top ISP

24-hour CFR% 3.20% 2.50% 4.50% Ameritech, PacBell (tie)

Evening-hour CFR% 3.80% 3.70% 5.70% Ameritech, PacBell (tie)

Business-hour CFR% 3.50% 2.10% 4.60% PacBell

Initial modem speed (Kbps) 47.65 48.07 48.04 AT&T, ICG-NetAhead (tie)

Avg. time to login (sec) 29.58 31.27 30.55 AT&T

Avg. DNS lookup (msec) 50979.00% 54747.00% 62747.00% AT&T

Avg. Web Tput (KB/s) 4.41 4.73 4.51 AT&T, PacBell (tie)

Avg. download time (sec) 25.64 26.06 28.17 AOL

Avg. total Web fail % 1.20% 1.00% 1.60% AT&T

(back to top)

How we did it

The data for this report comes from Visual Networks. Through its Internet Benchmark data, Visual rates national, regional and business-to-business Internet Service Providers according to several factors, including call failure rate, modem connect speed, and Web download performance.

We took the raw data from Visual and applied statistical analysis to rate the relative performance of each ISP. First, we derived the standard deviation of the numbers in each performance category. Standard deviation is a measure of how far the numbers in a series diverge from each other.

For each category in which an ISP performed better than one standard deviation from the industry mean, we award it one point. If the ISP did better than two standard deviations from the mean, we awarded it two points.

Similarly, if an ISP did worse than the industry mean by more than one standard deviation, we took a point away from its score. If it did significantly worse, meaning two or more standard deviations, we took away two points. We started with a baseline of 12 points, so if an ISP scores 0 points they end up with an adjusted score of 12.

After we scored each category, we summed the results to get a single number that indicates the reliability and performance of each ISP.

A rating of 12 means an ISP was about even with its peer, or the industry average. In November, eight ISPs scored higher than 12 and eight ISPs were average. The bad news, however, is that 11 ISPs scored less than 12 points. A total of 27 ISPs were tested.

A list of all the ISPs tested, as well as an archive of previous reports, appears below.

(back to top)

Quick Tips

DNS lookup

Throughput and connection speed aren't the end of the story when determining network performance. Domain Name System (DNS) performance adds additional overhead to almost any network activity. DNS translates host names like www.yahoo.com into IP addresses used by Internet routers. For example, when you open a Yahoo page, your browser requires at least one DNS lookup to complete before it can request the data for that page.

A typical DNS request is about 80 bytes in size, taking about 15 msec to squeeze through a 56K bit/sec line to your ISP. Responses vary, ranging from about 100 to 400 bytes in size and taking 15 msec to 70 msec to return. On top of this, each ISP's performance will differ based on how its DNS infrastructure is deployed.

The impact of DNS performance can be fairly complex. Local DNS lookup caches are maintained by Windows and other operating systems. These speed lookups when the same host name is requested repeatedly. DNS servers also maintain caches, ensuring that DNS queries for heavily used Web sites will often be answered quickly. In addition, each Web page may contain several objects from different servers - or all the objects could be from the same server, maximizing the effectiveness of a local cache.

(back to top)

ISP report archive:

Top ISPs, 12/04/00
Top ISPs, 10/23/00
Top ISPs, 09/25/00
Top ISPs, 08/28/00
Top ISPs, 07/24/00
Top ISPs, 06/26/00
Top ISPs, 05/29/00
Top ISPs, 04/17/00
Top ISPs, 01/31/00