Evaluating Cloud Computing Services

Selecting a cloud computing provider is becoming increasingly complex. As cloud environments mature, many cloud providers attempt to differentiate themselves by focusing on specific aspects of their offerings, such as technology stacks or service-level agreements (SLAs). In short, not all cloud providers are created equal.

At the same time, enterprises are beginning to rely on cloud providers for hosting mission-critical applications, which raises the stakes for selecting the right cloud service. So how do organizations navigate this multifarious landscape? Below you’ll find a few key factors for evaluating services as well as some resources to use.


One of the main concerns for enterprises that are considering cloud computing is performance. Achieving high-speed delivery of applications in the cloud is a multifaceted challenge that requires a holistic approach and an end-to-end view of the application request-response path.

Performance issues include the geographical proximity of the application and data to the end user, network performance both within the cloud and in-and-out of the cloud and I/O access speed between the compute layer and the multiple tiers of data stores. A number of services and research reports such as CloudSleuth and CloudHarmony have recently attempted to measure the performance of cloud providers from various locations and with different application use cases.

Technology stack

Several cloud providers have focused their services on a particular software stack. This typically moves them from being Infrastructure as a Service (IaaS) providers to the realm of Platform as a Service (PaaS). As one would expect, the different stack-specific clouds align with the most popular software stacks out there.
Examples include Heroku and Engine Yard for Ruby on Rails; VMforce and Google App Engine (GAE) for Java/Spring (GAE also supports Python), PHP Fog for PHP and Microsoft’s Windows Azure for .NET.

If your application is built using one of these stacks, you may want to consider these cloud platforms. They can offer tremendous savings in terms of time and expense by shielding you from having to deal with lower level infrastructure setup and configuration. The flip side is that they often require developers to follow certain best practices in architecting and writing their apps, which creates a higher degree of vendor lock-in.

Service-level agreements and reliability

Some cloud providers offer guarantees for higher levels of service as a way to separate themselves from the pack. In Rackspace: The Avis of Cloud Computing, I describe how Rackspace has higher levels of cloud service SLAs to compete with Amazon, the 800-pound gorilla of cloud computing.

Note that SLAs are often merely an indication of the consequences when the service fails and not the service’s actual reliability. A great example of this is GoGrid’s 10,000% Guaranteed SLA. In other words, GoGrid offers a 100% uptime guarantee. Should it fail to meet that level of availability, it will compensate the customer with 100 times the fee paid for the downtime.

Although the SLA is a good indicator of any provider’s level of commitment, knowing the real uptime levels of a particular cloud provider is a trickier proposition. Most vendors have a status page that acts as a dashboard for the health of their services, but these generally display only stats from a few days ago at the earliest. To get actual long-term numbers for reliability and availability, it’s better to rely on customer testimonials and comparison services such as CloudSleuth and CloudHarmony.

APIs: Lock-in, community and ecosystem

Another critical aspect of selecting a cloud provider is the application programming interface (API) it exposes for accessing the infrastructure and performing operations such as provisioning and de-provisioning servers. The API is important in a number of ways.

First, an API that is supported by multiple providers and vendors reduces lock-in because migration from one provider to another — or simultaneously working with multiple providers — requires less change to the application and is, therefore, easier.

Second, an API that is widely supported by a community of developers and vendors has an entire ecosystem around it of complementary services and capabilities. The APIs offered by Amazon Web Services (AWS) and the various VMware cloud offerings have large ecosystems built around them, which includes tools for governance (such as enStratus), monitoring and management (such as Cloudkick and RightScale) and a slew of other services that complete their cloud service.

VMware itself does not have a cloud service, but various providers use the VMware stack and APIs — specifically vCloud — such as Terremark and Savvis.

Both Amazon and VMware — and perhaps Windows Azure as well — allow customers to implement in-house clouds using their stack and APIs, thus enabling an easy way to manage and run applications on what some call a hybrid cloud. A hybrid cloud is a cloud that is both hosted by a provider and runs in the company’s on-premise data center. In the case of Amazon, this can be done through Eucalyptus, a startup that provides a software stack for implementing private clouds using the AWS APIs.

A recent development in the space is worth mentioning. Rackspace, jointly with NASA and supported by many vendors and cloud providers, has open sourced its software stack in a project called OpenStack. Along with being the closest to what might be considered an industry standard, this move creates a viable alternative to the Amazon and VMware ecosystems.

Security and compliance

Two of the biggest barriers for companies considering cloud computing continue to be security and compliance. In a recent Zenoss Inc. survey conducted during the second quarter of 2010, nearly 40% of respondents listed security when asked about their biggest concerns about cloud computing. The second most common answer was management, which received only 26.5% of the responses. The Zenoss survey is consistent with a number of other surveys related to cloud computing.

The real concern for enterprises is not actually security threats but rather their inability to achieve compliance with security-related standards such as PCI. In response, many cloud providers are now touting their security and compliance chops with SAS-70 Type II audits, security white papers and other measures.

Banking on the opportunity, cloud provider Logicworks, has dubbed its offering the Compliant Cloud and has recently announced the Level 1 PCI accreditation of its cloud.


A straightforward way to compare cloud providers would appear to be cost, but it turns out to be anything but. The problem is that there is no consistency among providers in regards to the resources customers actually receive and pay for. Providers offer virtual machines (VMs) that vary widely in memory capacity, CPU clock speed and other features. Furthermore, the units that are actually provided to customers are often virtualized, creating even further confusion as to what the customer is actually getting and how it might be affected by other customers on the same cloud.

Amazon has EC2 Compute Units, Heroku offers Dynos and other vendors have created their own measurement units. The only truly reliable way to measure the cost-performance of different cloud providers at this point is to conduct an experiment with the same application or prototype on multiple providers and compare the results.


Choosing the best cloud provider for an application is a multidimensional problem. As the number of cloud providers increases, and as many of them focus on specialized needs and use cases, more choices require more focused examinations. Fortunately, services are emerging that help compare cloud services so that customers can tell which provider is best suited for which application.


Like it? Share it! facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

26 thoughts on “Evaluating Cloud Computing Services

  1. Pingback: Tweets that mention Evaluating Cloud Computing Services - meshIP Blog -- Topsy.com

  2. Solange Nims

    Pretty excellent post. I just stumbled upon your blog and desired to say that I’ve really enjoyed reading your blog page posts. Any way I’ll be subscribing in your feed and I hope you post again soon.

  3. Leon Val

    Many thanks for your personal challenge plus efforts! These things against your webpage is definitely awesome. Also I actually greatly value a person’s tips. Personally these are typically essential matters. Nonetheless cheers. Fine look over.

  4. Benjamin Vanoort

    I actually like what you submit here. Highly insightful and intelligent. One problem though. I’m running Opera with Debian and parts of your current structure pieces are a little wonky. I realize it’s not a regular set up. Yet it’s an issue to keep in mind. I hope that it will help and continue to keep up the top notch quality writing.

  5. Deloris Solorsano

    This page appears to recieve a good ammount of visitors. How do you promote it? It offers a nice unique twist on things. I guess having something authentic or substantial to give info on is the most important thing.

  6. Donn Spratlin

    When i quite prefer whatever you submit in this article. Extremely useful in addition to smart. Just one challenge though. I’m running Internet explorer having Debian in addition to elements of this present web design portions undoubtedly are a very little wonky. When i understand it’s an excellent normal established. Yet still it’s an issue to help retain in view. When i trust so it can guide in addition to always keep in the prime excellent publishing.

  7. Bari Mccoulskey

    I’d come to give green light with you one this subject. Which is not something I typically do! I enjoy reading a post that will make people think. Also, thanks for allowing me to speak my mind!

  8. Trish Schoultz

    When i seriously enjoy whatever you post right here. Highly useful in addition to smart. Just one concern though. I’m running Firefox having Debian in addition to elements of this present page layout portions undoubtedly are a very little wonky. When i understand it’s an excellent regular established. But it’s something to help hold in the mind. When i wish so it will probably guide in addition to keep the best excellent publishing.

  9. Sherie Leyda

    Let me start by declaring wonderful post. Im not certain if it has been talked about, but when using Chrome I can never get the entire web site to load without having refreshing numerous times. Could just be my computer. Thanks.

  10. bola tangkas

    What youre stating is totally true. I understand that everybody need to say the identical issue, but I just believe that you put it within a way that everyone can understand. I also enjoy the images you put in here. They fit so well with what youre wanting to say. Im confident youll attain numerous individuals with what youve acquired to say.

  11. Genna King

    When i seriously enjoy whatever you post right here. Highly useful in addition to smart. Just one concern though. I’m running Firefox having Debian in addition to elements of this present page layout portions undoubtedly are a very little wonky. When i understand it’s an excellent regular established. But it’s something to help hold in the mind. When i wish so it will probably guide in addition to keep the best excellent publishing.

  12. shoppingcenter

    Excellent publish! I?m just starting out in community management/marketing media and looking to find out tips on how to do it nicely – resources like this post are incredibly helpful. As our company is based from the US, it?s all a bit new to us. The example above is some thing that I worry about as effectively, how you can show your own genuine enthusiasm and share the fact that your product is useful in that case

  13. penyakit lupus

    Have you ever considered adding more videos to your weblog posts to keep the readers more entertained? I imply I just go through through the entire write-up of yours and it was fairly good but since I’m more of a visual learner,I discovered that to be more helpful nicely let me know how it turns out! I adore what you guys are always up too. This kind of clever operate and reporting! Hold up the wonderful works guys I’ve added you guys to my blogroll. That is a wonderful article thanks for sharing this informative info.. I will check out your blog site regularly for some latest post.

  14. bielizna erotyczna

    Relatively fine put up. I actually really came across ones own weblog and even needed to help you express which will I’ve extremely loved checking ones own web site and even threads. Anyhow I’ll end up checking ones own rss feed and even As i expect to help you examine ones own web site all over again.

  15. Brandon Florin

    I like the helpful info you provide in your articles. I will bookmark your weblog and check again here frequently. I am quite certain I’ll learn plenty of new stuff right here! Good luck for the next! Brandon Florin

  16. Carmel Gutieres

    We’re a group of volunteers and starting a new scheme in our community. Your website offered us with valuable information to work on. You have done an impressive job and our entire community will be grateful to you. Carmel Gutieres

  17. Jennie Allen

    Wow! This could be one particular of the most helpful blogs We have ever arrive across on this subject. Actually Excellent. I’m also a specialist in this topic so I can understand your hard work.

  18. bezdech senny

    Really fine blog post. I actually simply came across any blog site and additionally wished to make sure you express which usually I’ve extremely appreciated examining any weblog and additionally threads. Anyhow I’ll come to be signing up any satisfy and additionally Document wish to make sure you examine any weblog once again.

Leave a Reply