Ultimate cloud speed tests: Amazon vs. Google vs. Windows Azure

1 2 Page 2
Page 2 of 2

The simplest option is Windows Azure, which sells machines in sizes that range from extra small to extra large. The amount of CPU power and RAM generally increase in lockstep, roughly doubling at each step up the size chart. Microsoft also offers a few loaded machines with an extra large amount of RAM included. The smallest machines with 768MB of RAM start at 2 cents per hour, and the biggest machines with 56GB of RAM can top off at $1.60 per hour. The Windows Azure pricing calculator makes it straightforward.

One of the interesting details is that Microsoft charges more for a machine running Microsoft's operating system. While Windows Azure sometimes sold Linux instances for the same price, at this writing, it's charging exactly 50 percent more if the machine runs Windows. The marketing department probably went back and forth trying to decide whether to price Windows as if it's an equal or a premium product before deciding that, duh, of course Windows is a premium. 

Google also follows the same basic mechanism of doubling the size of the machine and then doubling the price. The standard machines start at 10.4 cents per hour for one CPU and 3.75GB of RAM and then double in capacity and price until they reach $1.66 per hour for 16 CPUs and 60GB of RAM. Google also offers options with higher and lower amounts of RAM per CPU, and the prices move along a different scale.

The most interesting options come from Amazon, which has an even larger number of machines and a larger set of complex pricing options. Amazon charges roughly double for twice as much RAM and CPU capacity, but it also varies the price based upon the amount of disk storage. The newest machines include SSD options, but the older instances without flash storage are still available.

Amazon also offers the chance to create "reserved instances" by pre-purchasing some of the CPU capacity for one or three years. If you do this, the machines sport lower per-hour prices. You're locking in some of the capacity but maintaining the freedom to turn the machines on and off as you need them. All of this means that you can ask yourself how much you intend to use Amazon's cloud over the next few years because it will then help you save more money.

In an effort to simplify things, Google created the GCEU (Google Compute Engine Unit) to measure CPU power and "chose 2.75 GCEUs to represent the minimum power of one logical core (a hardware hyper-thread) on our Sandy Bridge platform." Similarly, Amazon measures its machines with Elastic Compute Units, or ECUs. Its big fat eight-CPU machine, known as the m3.2xlarge, is rated at 26 ECUs while the basic one-core version, the m3.medium, is rated at three ECUs. That's a difference of more than a factor of eight.

This is a laudable effort to bring some light to the subject, but the benchmark performance doesn't track the GCEUs or ECUs too closely. RAM is often a big part of the equation that's overlooked, and the algorithms can't always use all of the CPU cores they're given. Amazon's m3.2xlarge machine, for instance, was often only two to four times faster than the m3.medium, although it did get close to being eight times faster on a few of the benchmarks.

Caveat cloudsterThe good news is that the cloud computing business is competitive and efficient. You put in your credit card number, and a server pops out. If you're just looking for a machine and don't have hard and fast performance numbers in mind, you can't go wrong with any of these providers.

Is one cheaper or faster? The accompanying tables show the fastest and cheapest results in green and the slowest and priciest results in red. There's plenty of green in Google's table and plenty of red in Amazon's. Depending on how much you emphasize cost, the winners shift. Microsoft's Windows Azure machines start running green when you take the cost into account.

The freaky thing is that these results are far from consistent, even across the same architecture. Some of Microsoft's machines have green numbers and red numbers for the same machine. Google's one-CPU machine is full of green but runs red with the Tradesoap test. Is this a problem with the test or Google's handling of it? Who knows? Google's two-CPU machine is slowest on the Fop test -- and Google's one-CPU machine is fastest. Go figure.

All of these results mean that doing your own testing is crucial. If you're intent on squeezing the most performance out of your nickel, you'll have to do some comparison testing and be ready to churn some numbers. The performance varies, and the price is only roughly correlated with usable power. There are a number of tasks where it would just be a waste of money to buy a fancier machine with extra cores because your algorithm can't use them. If you don't test these things, you can be wasting your budget.

It's also important to recognize that there can be quite a bit of markup hidden in these prices. For comparison, I also ran the benchmarks on a basic eight-core (AMD FX-8350) machine with 16GB of RAM on my desk. It was generally faster than Windows Azure's eight-core machine, just a bit slower than Google's eight-core machine, and about the same speed as Amazon's eight-core box. Yet the price was markedly different. The desktop machine cost about $600, and you should be able to put together a server in the same ballpark. The Google machine costs 82 cents per hour or about $610 for a 31-day month. You could start saving money after the first month if you build the machine yourself.

The price of the machine, though, is just part of the equation. Hosting the computer costs money, or more to the point, hosting lots of computers costs lots of money. The cloud services will be most attractive to companies that need big blocks of compute power for short sessions. If they pay by the hour and run the machines for only a short block of time, they can cut the costs dramatically. If your workload appears in short bursts, the markup isn't a problem because any machine you own will just sit there most of the day waiting, wasting cycles and driving up the air conditioning bills.

All of these facts make choosing a cloud service dramatically more complicated and difficult than it might appear. The marketing is glossy and the imagery makes it all look comfy, but hidden underneath is plenty of complexity. The only way you can tell if you're getting what you're paying for is to test and test some more. Only then can you make a decision about whether the light, airy simplicity of a cloud machine is for you.

This article, "Ultimate cloud speed tests: Amazon vs. Google vs. Windows Azure," was originally published at InfoWorld.com. Follow the latest developments in cloud computing at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.

Read more about cloud computing in InfoWorld's Cloud Computing Channel.

This story, "Ultimate cloud speed tests: Amazon vs. Google vs. Windows Azure" was originally published by InfoWorld.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Related:
1 2 Page 2
Page 2 of 2
Take IDG’s 2020 IT Salary Survey: You’ll provide important data and have a chance to win $500.