Researchers ping through first full 'Internet census' in 25 years

With an assist from xkcd

No door-to-door canvassing here: This census involved the direction of some 3 billion pings toward 2.8 million allocated Internet addresses from three machines over the course of two months.

And, oddly enough, the researchers say their project – designed to help predict future trends and improve ‘Net security -- received data-presentation advice from an unexpected source: a comic strip, albeit one fittingly made popular by the Web.

(2012’s 25 Geekiest 25th Anniversaries)

(2011's 25 Geekiest 25th Anniversaries)

(2010's 25 Geekiest 25th Anniversaries)

Such a comprehensive census of the “visible Internet” had not been undertaken since 1982 when David Smallberg conducted a full accounting of what was then a a mere 315 allocated addresses (RFC 832), according to researchers at the University of Southern California Information Sciences Institute.

From the research group’s press release:

Many (61 percent) of the pings received no response at all. Many others got a "do not disturb" or "no information available" response that many network administrators program into their routers and firewalls. Some of the non- replies were probably also due to firewalls intentionally blocking the pings. Still, as the census went on, millions of sites did respond, positively and negatively, and a unique Internet atlas took shape.

The research comes amidst an ongoing and somewhat controversial transition from IPv4 and its dwindling supply of Internet addresses to IPv6, which backers tout as having a virtually inexhaustible supply.

Presenting the census results graphically was a major challenge, one which researchers met through the help of a popular Web cartoon.

These addresses appear in the chart as a grid of squares, each square representing all the addresses beginning with the same first number ("128," in the preceding example). The map is arranged in not in simple ascending numerical order, but instead in a looping pattern called a Hilbert curve, which keeps adjacent addresses physically near each other, and also makes it possible to zoom seamlessly in to show greater detail. "The idea of using a Hilbert curve actually came from a web comic, xkcd," said Heidemann.

Here’s what I assume is the xkcd strip in question.

And here’s a related discussion on the author’s blog from Dec. 11, 2006.

Wikipedia’s explanation of a Hilbert curve can be found here.

My dissertation on the subject … ah, never mind.

As for what is hoped to be accomplished through this research:

"Internet census data is useful for several reasons", Heidemannn says. "As the Internet use becomes widespread, we are running out of Internet addresses—good predictions by Geoff Huston suggest all addresses may be allocated as soon as early 2010. The IETF (Internet Engineering Task Force, the technical body that manages the Internet) has anticipated this since the 1990s and designed a new protocol, IPv6, to solve this problem, but deployment has been slow. Our data can help illustrate the need to move forward."

It's hoped that the census also can improve Internet security. In fact, the Department of Homeland Security "supported our work with the goal of improving network security," said Heidemann, pointing to the work of ISI researcher Jelena Mirkovic that is using this census data to study how worms spread in the Internet. Other researchers have plotted maps of where cyber-attacks originate.

Additional information – and all the pretty pictures – can be found at the main Web page for the census project and the related “Plotting the Whole Internet.”

Welcome regulars and passersby. Here are a few more recent buzzblog items. And, if you’d like to receive Buzzblog via e-mail newsletter, here’s where to sign up. You can follow me on Twitter here and on Google+ here.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2007 IDG Communications, Inc.

SD-WAN buyers guide: Key questions to ask vendors (and yourself)