Skip Links

Network World

  • Social Web 
  • Email 
  • Close

Open source tool emerges for managing petabytes of data

UC San Diego supercomputing group airs iRODS data grid software
By Network World Staff , Network World , 02/11/2008

The University of California at San Diego's supercomputing group has upgraded its time-tested data management software and made it available as an open source offering designed to handle up to petabytes of data.

The Data-Intensive Computing Environments (DICE) group at the San Diego Supercomputer Center (SDSC) has issued iRODS (Integrated Rule-Oriented Data System) 1.0, which the outfit says improves on and goes beyond the Storage Resource Broker it has developed over the past 10 years.

"iRODS equips users to handle the full range of distributed-data-management needs, from extracting descriptive metadata and managing their data to moving it efficiently, sharing data securely with collaborators, publishing it in digital libraries, and finally archiving data for long-term preservation," said Reagan Moore, director of the DICE group, in a statement.

The heart of iRODS is a rules engine that supports customized data-management needs, such as retaining records in one place for a set period of time, then automatically destroying them. (Compare Storage Resource Management products.)

While iRODS is simple enough to be used by an individual user, at the SDSC it is used to handle a petabyte of data and 200 million files for 5,000 users, according to the university. The center, by the way, has won a best-practices award for the design of its new data center that will help to make it a model of energy efficiency.

The National Archives and Records Administration (which funded the project along with the National Science Foundation) and Ocean Observatories Initiative are among other iRODS users.

Version 1.0 of iRODS runs on Linux, Solaris, Macintosh and AIX, with Windows support on the way. It runs on the open source PostgreSQL database and Oracle databases, and works across multiple physical servers.

The DICE team is offering tutorials and workshops for interested users.

For more on network research, visit our Alpha Doggs blog.

Partner Content

Explore the Ultrium Edge

The powerful tape technology can address data security with tape encryption as well as long term data protection.

Find out more

Disk and Tape Square Off

Discover what disk and tape really cost -- and which solution provides lower total cost of ownership and optimizes energy use for your organization

Download the White Paper

Don't Fall For The Myths

The Clipper Group explores the truth behind the myths of tape, digging into the misconceptions in the disk vs. tape debate.

Download the White Paper

Will You Add Tape Too?

Over two thirds of disk-only users look to add tape back into storage infrastructure according to recent survey.

Download Survey Information

Comment
Login
Forgot your account info?
Add comment
Anonymous comments subject to moderator approval. Register here for member benefits.
Have a NetworkWorld account? Log in here. Register now for a free account.

Videos

rssRss Feed
Save The Date!
What They Are Saying

In all of these letters that you have posted, Chuck, I have yet to see one that apologizes to PZ Myers...- bullet

Join the Discussion