This company is using Amazon Snowmobile to transfer petabytes of data to the cloud

AWS's Snowmobile is a data center on wheels that uploads data 100 Petabytes at a time to the cloud

AWS Snowball Amazon DigitalGlobe
IBM, Maersk

One of the most dramatic announcements from Amazon Web Services at its 2016 re:Invent conference was the announcement of Snowmobile: It’s a 45’ semi truck that trailers a data center on wheels. Customers can load it up with up to 100 petabytes of data per Snowmobile, which is then driven to an AWS data center and loaded into the company’s cloud.

It begs the question: Who’s actually using this? DigitalGlobe (DGI) is one company.

+MORE AT NETWORK WORLD: Amazon will literally truck your data into its cloud

DGI has a fleet of satellites that collect geospatial images of the earth. It’s a publicly traded company has been around since 1992, so in that time it has amassed a lot of data: about 100 petabytes worth, growing at about 10 PBs per year. DGI built up its own IT infrastructure to deal with this data but in recent years has embraced the cloud. “Who wants to be moving around petabytes of data and who has the compute capacity to do analytics on that data at massive scale?” says Walter Scott, DGI’s Founder and CTO. “Rather than bring the data to our customers, we’re moving the data to the cloud.”

DGI uses AWS’s Snowballs, which are 50 TB or 80 TB hardware devices that can be loaded up and sent to AWS for upload to the cloud. Transferring DGI’s full collection of data in 80TB increments would take up to four years though, Scott says.

That’s why DGI is one of the first customers to use the Snowmobile. There’s one parked outside of the company’s data center where its being loaded up over the course of many weeks. When it’s full, it’ll be driven to an AWS data center where the data will be uploaded into a combination of Amazon’s Simple Storage Service (S3) – the scalable object storage service, and Glacier, the ‘cold’ storage service for more inexpensively storing data that is less frequently accessed.

What does DGI do with all this data anyway? It’s built a geospatial big data platform named GBDX which uses machine learning, crowd sourced information and huge amounts of computing power to analyze the geospatial imagery and run queries against it. Humanitarian efforts can use it to determine which areas in a disaster most need resources; scientists use is to track environmental changes, such as the number of trees in an area; it’s even being used to help determine flight paths for drones. DGI’s goal is to be able to sell the GBDX platform to companies who will build tailored solutions for customers across many industries. DGI hopes to offload the heavy-lifting of collecting the imagery so its customers can have access to it whenever they need it. At the same time, DGI is offloading the compute and storage for all that data to AWS. And to get it all there, they’re using the Snowmobile.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Related:
Take IDG’s 2020 IT Salary Survey: You’ll provide important data and have a chance to win $500.