Grid surprise

Once considered a specialty technology, the latest buzz pegs grids as great all-around application servers.

As grid computing enters more enterprise environments, the buzz over the technology's potential never ceases. Once grids are installed, network executives find them useful for a far wider variety of applications than just computationally heavy ones. They also work well for applications that have high transactional volumes or are data intensive. And after sending those apps to the grid, it dawns on these early adopters that what they have is a giant, powerful - and comparatively inexpensive - next-generation generic application server.

"We have more than 700 nodes on our grid. Grid is our virtual application server that includes Windows, Linux and Solaris/Sun applications. We do have [dedicated] application servers, but our basic plan is to use the grid to provide a very viable alternative to expensive Sun platform boxes," says Robert Ortega, vice president of architecture and engineering for Wachovia, in Charlotte, N.C.

As it turns out, the "application" that a grid is almost innately suited to serve is none other than the industry's next-generation application itself - Web services .

"There's no point in moving an entire organization onto a , but host and provision those services in fixed, static [hardware] silos - where each application or service is run in its own separate environment, with its own personnel. If I'm going to marry my vision of loosely coupled services to an on-demand environment, I'm going to assume that the grid will allocate and provision [hardware] resources on demand as my service levels fluctuate," says Peter Lee, CEO of grid middleware vendor DataSynapse. "Grid is a very logical underpinning to achieve an SOA."

Ortega agrees. "Originally, the grid was only used for compute-intense applications, but now we're positioning it as a general-purpose transactional environment," he says.

Wachovia built its grid in 2002, using software from DataSynapse (a company in which Wachovia's venture unit has invested, according to Wachovia materials). Wachovia has been using the grid to crunch numbers for risk analysis, securities prices and other applications, and to crank out enormous reports - a 15-minute process that used to take 15 hours, Ortega says.

Last year, IT executives expanded use of the grid, evolving it into an SOA platform that runs various Web services. For instance, Wachovia runs what Ortega calls "transformation" Web services on the grid. These are transactional messages that must be converted from one format to another, or that call on an application rule. Ortega uses Web services as the middleware between APIs. A Web service is used, for example, when an application requires data to be converted into a Java JAR file or a Windows DDL. By using a Web service to do the conversions, data can be passed from one proprietary application to another without custom API work. And because Wachovia's systems can field some 5,000 transformation messages per second, the grid's nearly unlimited power makes such an application design feasible, Ortega says.

In fact, the grid is the server of choice for "any application that somebody's trying to build here that uses a J2EE application server," he says, adding that to the run-time version of the Java app the grid looks no different than a stand-alone server. "We keep it virtualized, meaning that apps don't connect to a specific set of hardware, but to an abstract layer." This abstract layer is the DataSynapse software that distributes the workload among the available CPUs on the grid, collects the results and hands them back to the application - as if the underlying CPU was a server and not a grid. Ultimately, Ortega envisions that this giant application server will become the basis for utility computing at Wachovia and for converting IT into a managed service model where departments are charged for their use of the utility.

Wachovia, while perhaps further along than most businesses using grid, is hardly alone in its view of grid as a generic application server rather than a specialized server that's not suitable for many applications, says grid guru Ian Foster, director of grid technologies for Argonne National Laboratory and a founder of grid services firm Univa (a start-up to watch for 2005) .

"Web services on grid computing is being well received," he says, crediting the number of Web services/grid standards for such progress (see below). "What's been somewhat of a surprise to [the grid community in the last year] is that grid has become very much the mainstream infrastructure - grid is what makes it possible to build flexible service-oriented architectures."

8 questions to ask concerning grid computing Grid software vendors: What software has been demonstrated to operate on your grid infrastructure? What do I need to do to my applications so they can be used across the grid? What support will I be given to work with my current enterprise application vendors to move software to the grid? What interoperability tests have been done with your software and other virtualization products? Application vendors: Is anyone running this software on a grid? Is anyone running this software using data virtualization? Can this software be modified so that its computational needs are divided among a large multiprocessor system? What are you willing to do to adjust your software so that it can be operated on grid?

Rewrite those apps

Early grid adopter Terry Talley, a senior engineer for Acxiom, also finds grids to be natural as SOA application servers. He sees services in an SOA as inclusive of both cutting-edge Web services and older technologies such as Common Object Request Broker Architecture (CORBA). Acxiom, a data warehouse/data mining system integrator in Little Rock, Ark., built its grid in 2001 to handle the massive amounts of data cleaning needed to convert client archives into functional warehouses. Since then, the grid has become a foundation for nearly all of Acxiom's data warehouse services.

"We have a lot of [Web services and CORBA] on our grid - and that's not an accident. We require massive throughput ... millions of records per hour," he says.

Grid and Web services standards bodies Enterprise Grid Alliance: A consortium of vendors and others that have created a reference model for grid in the enterprise. Global Grid Forum: A scientific, open source and user community defining a series of standards for its Open Grid Services Architecture. Globus Alliance: A consortium of mostly academicians and scientists creating an open source development tool kit for building grids. OASIS: This Web services standards body approved the Web Services Distributed Management standard (in March 2005) for defining management of Web services in a highly distributed environment. It is working on other standards, including WSResource Framework.

But Talley offers one strong caveat - most applications will need to be re-architected to run on a grid. "In some cases, you can deploy software on grid with little or no change, but that's not typical. To really exploit the grid, you want to look at how your applications can exploit large amounts of CPU," he says.

Applications that consider CPU as a nearly endless resource are rare because of the IT industry's traditional mainframe orientation, Talley says. Application developers have been taught to go to extremes to shove lots of applications on the server "and squeeze everything they could out of that box," he says. With grid, the reverse is true. CPU resources are nearly endless. An application has to be able to be split apart into lots of tiny processes, it has to be functional on a multiprocessor design, and it has to be tough enough to handle delays in some parts of its processes.

Ortega agrees, but adds that because Wachovia's grid has virtualized the application layer with transformation Web services, the financial services giant has made it easy for its strategic vendors to step up and re-architect their software. "All the vendor has to do is generate a model [where code is] packaged in a .dll or Java class and DataSynapse does the rest. Once we deploy that [.dll or Java class] to the grid, then that piece of code appears as a pure Web service," he says.

At least some enterprise application vendors are toying with grid-friendly designs. SAP, for example, has publicly demonstrated grid versions of its CRM and supply chain management (SCM) software. SAP - the poster child for the monolithic applications - carved out portions of its big centralized system so that they could be run on multiple servers using the open source Globus tool kit as the grid infrastructure, says Foster, one of the creators of the tool kit. "When the app said, 'I need another 10 servers to meet my workload demands,' it reached out to the [grid] infrastructure to meet the demand," he says.

These demonstrations were especially significant for enterprise grid computing because they showed portions of the CRM and SCM software complying with , the grid/Web services standard, Foster says. And that was a significant move to prove that any business app can run on a grid, he concludes.

For network executives, it also might be a message about your next application server computing platform: Think big.

Grid and virtualization The following types of virtualization can be helpful when using a grid as the application server for a service-oriented architecture. Application/business logic virtualization, such as Web services, where the business logic can be run on any type of underlying application server, even grid computers. Data virtualization, in which the grid would connect to an abstract layer that then retrieves data from any repository. I/O virtualization, in which the abstract layer separates the grid from storage systems. Operating system virtualization, in which the operating system is separated from the grid hardware.