Yahoo offers free supercomputing to Hadoop developers - Network World

Skip Links

DNSstuff.com
Get information about your IP
IP Information
50+ On-demand DNS and network tools

Data Center

Videos

rssRss Feed
Get instant email notification when white papers, webcasts, executive guides are added to our library.  Stay informed and up-to-date with the latest on IT Technologies with Network World's Resource Alerts.
Audio

Brocade acquires Foundry; Implicit Networks lawsuit. Listen now!

Network World 360

Virtualization helps whittle disaster recovery time from weeks to hours. Listen now!

Voices from IT Roadmap

Additional Resources

RSS

FEATURED WHITEPAPERS

Tuning ERP and the Supply Chain for Profitable Growth Epicor Software Corporation

The supply chain is, of course, the primary processing mechanism of every manufacturing company. But today, it is a multifaceted, multi-company, multinational structure that makes it the most complex management challenge found in any enterprise. Supply chain management (SCM), a critical part of any enterprise resource planning (ERP) system, no longer means just making sure that the right materials and resources move to the right place at the right time. It also means ensuring that the sequences of events involved in producing goods and distributing them to customers are tuned optimally. This white paper will discuss how ERP can optimize your supply chain to satisfy customers, minimize costs and maximize profit.

RSS

FEATURED REPORTS

Executive Guide: Storage Heats Up Overland Storage, Inc.

Learn the latest on promising storage technologies that allow IT professionals to better cope with new IT demands. Find out how storage technologies can help you successfully tackle e-Discover, regulatory compliance, green data center initiatives and the data explosion. Get all of the details now.

IT Buyer's Guides

View All Buyer's Guides

Free Newsletters

Sign up and receive the latest news, reviews and trends on your favorite technology topics

Save The Date!
What They Are Saying

If the IT manager is knowledgeable regarding Cisco technology, he would have 2 options. Option 1 - Consult...- Anonymous

Join the Discussion

Partner Content

Explore the Ultrium Edge

The powerful tape technology can address data security with tape encryption as well as long term data protection.

Find out more

Disk and Tape Square Off

Discover what disk and tape really cost -- and which solution provides lower total cost of ownership and optimizes energy use for your organization

Download the White Paper

Don't Fall For The Myths

The Clipper Group explores the truth behind the myths of tape, digging into the misconceptions in the disk vs. tape debate.

Download the White Paper

Will You Add Tape Too?

Over two thirds of disk-only users look to add tape back into storage infrastructure according to recent survey.

Download Survey Information

Yahoo offers free supercomputing to Hadoop developers

By John Ribeiro , IDG News Service , 03/25/2008
  • Social Web 
  • Email 
  • Close

Yahoo has teamed with Computational Research Laboratories (CRL), a lab run by India's Tata Group, to offer supercomputing facilities free to academic institutions in India that are researching large scale computing, particularly around Apache Hadoop, an open source distributed computing project of the Apache Software Foundation that Yahoo supports.

The deal is similar to one that the company completed last year with Carnegie Mellon University, providing Hadoop researchers at the university with a 4,000-processor supercomputer.

Yahoo aims to get more developers to research and develop applications that can scale around Hadoop, said Rajeev Rastogi, vice-president and head of Yahoo Labs Bangalore, in an interview on Tuesday. To this end the company plans to replicate the model it is trying in India in some other countries as well.

Educational institutions in India will have access to the supercomputer facilities of the CRL for their research for an unlimited period, according to Rastogi. "Our objective is to encourage long term projects," he said. Academic institutions typically do not have access to hardware and software infrastructure such as a giant cluster of thousands of nodes, he added.

There are a number of educational institutions in India that are already doing research in the area of large scale computing, and Yahoo will try to get some of them to work around Hadoop, according to Rastogi. The company may even sponsor research at some of these academic institutions, he added.

CRL's supercomputer is called the EKA, and is ranked the fourth fastest supercomputer in the world in the Top 500 Supercomputer list. The supercomputer is available for use on commercial terms.

EKA has 14,400 processors, 28T bytes of memory, 140T bytes of disk storage, a peak performance of 180T flops (floating point operations per second, and sustained computation capacity of 120T flops. EKA will run the latest version of Hadoop and other Yahoo supported, open-source distributed computing software such as the Pig parallel programming language developed by Yahoo Research.

Comment
Login
Forgot your account info?
Add comment
Anonymous comments subject to moderator approval. Register here for member benefits.
Have a NetworkWorld account? Log in here. Register now for a free account.
First Name
Last Name
E-mail
Zip Code