Sun has long held that the network is the computer. Today both commercial and private entities are proving the maxim true.
Pixar, the California-based animation house known for such movies as "Toy Story" and "A Bug's Life," is the poster child for parallel computing. For "A Bug's Life" the company utilized some 150 high-end Silicon Graphics machines that worked in unison to bring the film's characters to life.
The dual-processor Octane workstations and Onxy2 visual systems used to create the detailed images combined to form a massive parallel-processing computer. However, 150 Silicon Graphics machines do not come cheap.
In a similar parallel processing effort a group of enthusiasts linked by the Internet managed to create a global computer 24 times more powerful than Pixar's rendering farm using the idle cycles of everyday PCs - at a minimal cost.
Distributed.net, a non-profit organization of computer hobbyists, uses this idle cycle time to do its bidding - cracking encoded plaintext messages as part of a contest.
Typically, a standard PC sits idle most of the time. Surfing the Web, checking e-mail, using Word and Excel does not exert much pressure on a Pentium II processor that can handle 200 million instructions per second. Most of the time, the processor sits idle, waiting for something to do.
Distributed.net plans to take the unused cycles of everyday computers that are connected to the Internet and create a globally distributed supercomputer. Such a system can be used to work on complex projects that require massive amounts of parallel computing power without the need for a lot of communication bandwidth. In the past, such projects needed to book time on supercomputers to run their processes - an expensive proposition. Today, projects can farm parcels of work out to PCs, Macs and Unix boxes that would otherwise be idle.
Buoyed by its recent success in decoding a Data Encryption Standard (DES) message in less than 24 hours, Distributed.net now has some 100,000 computers working on its various projects at any given time, according to one of the group's founders, David McNett.
The group started in March of 1997 when it set out to crack the RC5-56 encryption standard in a distributed manner. They succeeded in 250 days, winning an RSA Data Security-sponsored prize of $10,000. Soon after the win, Distributed.net was officially founded as a nonprofit organization and began its next project: an assault on the RC5-64 encryption standard as part of another RSA-sponsored contest.
Though the RC5-64 project (known affectionately as "Bovine" by members) is ongoing, the group managed to win two other DES-related contests in its spare time - one in 1998 when it decrypted a DES-encoded message in 40 days and its most recent effort in January. The group failed to win a similar contest last summer as it was beaten to the prize by a computer specially built and operated by the Electronic Freedom Foundation known as "Deep Cracker." (Deep Cracker participated with Distributed.net in the latest DES contest and was the machine that uncovered the winning key.)
Distributed.net's architecture is elegantly simple in design, making it easy for anyone, not just technical types, to participate. Anyone with a connection to the Internet can download a special client from Distributed.net that runs in the background of his or her computer. Clients are available for all the major platforms, including Windows, Macintosh, most Unix flavors, including Linux. Those with old Amiga computers can also participate.
The group recruits participants via Internet relay chat and word of mouth.
Clients are assigned blocks of work by 1 of 22 keyservers located around the world. Keyservers receive their marching orders from a central keymaster, which oversees the entire project.
The keymaster is a mini-tower PC with a K6 300-MHz chip and 250M bytes of memory running FreeBSD, a Unix variant. Custom-built software tracks all keys that have been assigned and processed. Baker said the system is actually underloaded and runs the client software in its spare time as well.
Distributed.net uses proprietary protocols that utilize TCP packets via port 2064 for connecting its parts to help minimize the amount of bandwidth needed for communications. The total amount of data being transferred is minimal, especially on the client side, according to Daniel Baker, chief operating officer of Distributed.net. "The [client] protocol has very little overhead, so it's around 150 to 200 bytes of data per block [being transferred]," Baker says.
The keyserver-to-keymaster connection uses a different protocol designed for larger block transfers. The protocol also helps the keymaster keep tabs on each of the keyservers, Baker says.
Each block, in the case of the RC5-64 project, consists of between 268 million and 2.1 billion keys. In their "spare" time, computers crunch away at each block, looking for the elusive key that will uncover the encoded plaintext message.
The system uses a method called "brute force," meaning it checks each possible key in succession until the correct one is found. With RC5-64 there are 2 to the 64th possible keys - a number 32 digits long. A lot of processing power is needed to plow through all the keys.
Clients running in the background are nearly undetectable and do not seem to slow a computer's normal operation down. Network World tested one of Bovine's RC5 clients on a Pentium 166-MHz machine with 48 megabytes of RAM that was running Windows 95. Running without someone at the keyboard (for example, overnight), the machine processed an average of 230,000 keys per second. In full operation (five applications running, with the user switching between applications regularly) the machine calculates approximately 195,000 keys per second, without perceivable performance degradation.
Multiply one computer's key rate across the thousands of systems working in Distributed.net and one gets a pretty powerful computer. As of Feb. 1, Distributed.net was processing an average of 24 billion keys per second over the life of the 468-day project.
"Our system allows the average person, who cannot purchase some outstanding machine like a CRAY, to combine the power of their machine with other people's as a means of handling extraordinarily complex problems," Baker says.
Distributed.net is not alone in the quest to harness idle CPU cycles. There are a number of other projects ongoing or near start-up that hope to borrow a little CPU time from the average person.
Math enthusiasts can help in the search for the 40 trillionth bit of Pi and the quadrillionth bit of Pi. (Pi is the relationship between a circle's circumference and its diameter.) One can even help find prime numbers (numbers divisible by only one and themselves) as part of the Great Internet Mersenne Prime Search (GIMPS) project.
Mersenne Primes are named after Frenchmen Marin Mersenne and are defined as prime numbers that can be calculated by the formula 2 to the nth minus1.
George Woltman heads up the GIMPS project from his home in Orlando. "I have always been interested in recreational mathematics," Woltman says. "So when I retired I decided to pursue this a little more."
Woltman wrote the mathematics code and fellow enthusiast Scott Kurowski handled the networking code to handle the client/server end of the project.
GIMPS uses the same architecture as Distributed.net. A client is downloaded and a central server parcels out work.
"I see this as a way to do science and math research on the cheap," Woltman says. "If you have a project and can't afford a multi-million dollar supercomputer, this is an inexpensive way to work on your project."
GIMPS has resulted in three new Mersenne Primes to date, with the latest being over 909,526 digits long (2 to the 3,021,377th minus 1). A 200-MHz Pentium computer owned by 19-year-old Roland Clarkson, a student at California State University Dominguez Hills found the number. Anyone can be a winner just by donating his or her unused cycles.
Another teenage college student is responsible for the PiHex project. Colin Percival is a 19-year-old, fourth-year math major at Simon Fraser University in British Columbia.
Currently limited to Windows client machines, PiHex parcels out its work via e-mail. Participants download a client from the project's Web site (http://www.cecm.sfu.ca/projects/pihex/index.html), then send e-mail to Percival requesting a block of work. Once the computer is finished chewing on the assigned block of work, the results are returned to Percival via e-mail. Percival said he is working on automating the process as well as porting the client software to other platforms.
Commercial value?
In posts to an RC5 mailing list administered by Distributed.net, some members wonder if there is a commercial project that can be accomplished by the network's computing power. After all the group demonstrated the ability to generate more processing power than Pixar.
Distributed.net members debate whether they should try and put together a small animated film using their computing power. While bandwidth would be an issue-many members connect to the Internet via a standard 28.8K bit/sec modem - computer-processing power would not.
Part of the problem for such commercial ventures would be timeliness. While Pixar had complete control of its infrastructure and knows exactly how much computing power it has at its disposal; architectures such as Distributed.net's have no guaranteed output. People could get bored and turn off their machines, or turn to other projects.
Not all processor-intensive projects are suited for such an environment, either. To work, a problem must be divided into small bits that can be distributed and computed on separately, without needing interaction with other calculations.
Rick Villars, director of network software research at International Data Corp. in Framingham, Mass., sees a day when processing power could be rented out. However, Villars sees processor-cycle renting happening more at a business or industry-group level than at the individual consumer level.
Percival would like to see such parallel processing power put towards something like weather forecasting. Everyone can use and understand an accurate weather forecast.
"No matter what view you take on the usefulness of computing bits of Pi, cracking codes, or searching for ET, you have to admit that it is more useful than just wasting CPU cycles," Percival says.
RELATED LINKS
Great Internet Mersenne Prime Search Web site
DES code cracked in record time
Network World, 01/20/99
