Startup RStor promises a new type of distributed compute fabric

A startup funded by Cisco and featuring some big-name talent has come out of stealth mode with the promise of unifying data stored across multiple distributed data centers.

RStor is led by Giovanni Coglitore, the former head of the hardware team at Facebook and before that CTO at Rackable Systems. The company also features C-level talent who were veterans of EMC’s technology venture capital arm, Amazon Web Services, Microsoft, Google, VMware, Dropbox, Yahoo, and Samsung.

Bouyed by $45 million in venture capital money from Cisco, the company has announced RStor, a “hyper-distributed multicloud platform” that enables organizations to aggregate and automate compute resources from private data centers, public cloud providers, and trusted supercomputing centers across its networking fabric.

“We’re trying to solve the dilemma of everyone’s desire to have flexibility and visibility into their cloud activities,” Coglitore told me. “There is somewhat of an anarchy of information about cloud. The cloud to most is a confusing space. What we attempted to do is bring clarity to that.”

How RStor works

With RStor, customers have a low latency fabric connecting their data centers to compute providers, including supercomputer centers. The fabric is very fast, about 40ms between the East Coast and West Coast of the U.S.

It works on-premises, with cloud providers and with supercomputer centers, and is available through data center giant Equinix, although the company emphasizes it is data center-neutral and is working on signing up more data center providers.

The platform is designed to give users freedom over where they store data and what computing resources they use, without having to move data around for it to be processed. In theory, you can have data sitting in an East Coast data center and use a data center on the West Coast to process it without having to move the bits.

A unique element to RStor is it does not limit compute to just x86 processors. It supports compute providers with computing powered by x86, GPU, ARM, IBM Power and Cray.

Supercomputers that data centers can access

The access to supercomputer centers is also a first. Those supercomputer centers include the San Diego Supercomputing Center, Texas Advanced Computing Center, the Pittsburgh Supercomputer Center, and the New Mexico Consortium.

It does this through a technology called Singularity, first developed by the Department of Energy before being spun off as a private firm. It offers a containerized app that lets untrusted users operate in trusted environments. Before Singularity, there was no viable way for an outsider, like a private firm, to get into supercomputer centers via the cloud because VMs and Docker didn’t have the security necessary to allow untrusted users into those systems.

However, Singularity has been fully embraced by the HPC community, with more than 25,000 compute environments using it, said Coglitore.

RStor also uses machine learning to predict the best compute partner for a specific workload, so the more the system is used, the more intelligence it gains over the best location to do the processing on a specific task. Its analytics examine the cost and performance attributes of the task and compare them to the attributes of each cloud provider and deployment platform to determine the best place for the workload.

In addition to using their own data center, RStor customers have the option of uploading their data to RStor’s own data lake, which is in multiple Equinix data centers.

RStor also has an app store of batch-oriented, AI-focused apps that run on its fabric. These include Google TensorFlow, the Chainer AI framework, and molecular dynamics like Amber and VASP.

Coglitore described how one alpha test customer uses the fabric. It’s a biopharma company he did not identify that bridges five data centers in the U.S. and Europe.

“By attaching to our fabric, we gave them low latency access to those facilities and a new way to interact with distant facilities. They can stitch five data centers together and allocate underutilized compute cycles to other data centers that were overstretched. That allowed us to balance the haves and have-nots,” he said.

RStor is a subscription-based service that charges by acting as a broker between facilities and as a middleman with customers and supercomputer centers, plus the app store. It is entering into invitation-only beta and plans for general availability by the end of summer.