IT Management and Cloud Blog

« | Home | »

Demystifying Clouds

By John | February 5, 2008

force ma·jeure – noun Etymology: French, superior force
Date: 1883 1 : superior or irresistible force 2 : an event or effect that cannot be reasonably anticipated or controlled — compare act of god

I’ll admit it: I am caught up in the cloud hype. The caveat, however, is that I truly believe that this is disruptive technology. In this article, I am going to try to demystify some of the hype around utility cloud computing and focus in on the companies that are providing cloud solutions and the technology components that they are using. By no means am I professing to be an expert on this subject. My only intent is to render what I have learned thus far in my quest to understand cloud computing.

The Myths

Cloud computing will eliminate the need to IT personnel.

Using my 30 years of experience in IT as empirical proof, I am going to go out on a limb and suggest that this is a false prophecy. One of my first big projects in IT was in the 1980s, and I was tasked to implement “Computer Automated Operations.” Everyone was certain that all computer operators would loose their jobs. In fact, one company I talked to said that its operators were thinking of starting a union to prevent automated operations. The fact was that no one really lost his or her jobs. The good computer operators became analysts, and bad ones became tape operators.

There will only be five super computer utility like companies in the future.

Again, I will rely on empirical data. I have been buying automobiles for as long as I have been grinding IT, and all one has to do is look at the automotive industry’s history as a template to falsify this myth. Some clever person will always be in a back room somewhere with an idea for doing it better, faster, cheaper, and cleaner. In all likelihood, there will probably be a smaller number of mega-centers, but it is most likely that they will be joined by a massive eco-grid of small-to-medium players interconnecting various cloud services.

The Facts

Since cloud computing is in a definite hype cycle, everyone is trying to catch the wave (myself included). Therefore, a lot of things you will see will have cloud annotations. Why not? When something is not clearly defined and mostly misunderstood, it becomes one of god’s great gifts to marketers. I remember that, in the early days of IBM SOA talk, IBM was calling everything Tivoli an SOA. So I did a presentation at a Tivoli conference called “Explaining the ‘S’ in SOA and BSM.” Unfortunately, one of IBM’s lead SOA architects, not Tivoli and not a marketer, was in my presentation and tore me a new one. I was playing their game, I forgot that it was “Their Game.” Therefore, in this article I will try to minimize the hype and try to lay down some markers on what are the current variations of all things considered clouds.

Level 0

As flour is to a cookie, virtualization is to a cloud. People are always asking me (their first mistake) what is the difference between clouds and the “Grid” hype of the 1990s. My pat answer is “virtualization.” Virtualization is the secret sauce of a cloud. Like I said earlier, I am by no means an expert on cloud computing, but every cloud system that I have researched includes some form of a hypervisor. IMHO, viirtualization is the differentiator between the old “Grid” computing and the new “Cloud” computing. Therefore, my “Level 0″ definition for cloud providers is anyone who is piggy-backing, intentionally or un-intentionally, cloud computing by means of virtualization. The first company that comes to mind is Rackspace, which recently announced that it is going to add hosting virtual servers to their service offering. In fact, it new offering will allow a company to move its current in-house VMware servers to a Rackspace glass house. A number of small players are producing some rain is this space. A quick search on Google will yield monthly plans as low as $7 per month for XEN VPS hosting. It’s only a matter of time before cloned Amazon EC2 providers start pronouncing themselves as Cloud Computing because they will host XEN services in their own glass house. These services will all be terrific offerings and will probably reduce costs, but they will not quite be clouds, leaving them, alas, at “Level 0.”

Level 1

My definition of “Level 1″ cloud players are what I call niche players. “Level 1″ actually has several sub-categories.

Service Providers

Level 1 service provider offerings are usually on-ramp implementations relying on Level 2 or Level 3 backbone providers. For example, a company called RightScale un-mangles Amazon’s EC2 and S3 API’s and provides a dashboard and front-end hosting service for Amazon’s Web Services (AWS) offering (I.e., EC2 and S3). AWS is what I consider a “Level 2” offering, which I will discuss later in this article.

Service Hybrids

Service Hybrids are players like ENKI and Enomaly. Both companies offer services around backbone cloud providers in the form of services and software. In fact, I was baptized in the clouds by Reuven Cohen, the founder of Enomaly, on a plane ride from Austin to Chicago. I sat next to Reuven, and he was gracious enough to school me on Amazon’s AWS. Enomaly offers services and software around Amazaon’s AWS, and they are clearly the go-to guys for EC2/S3. ENKI seems to be Enomaly’s equivalent but with the 3Tera/Applogic application. 3Tera is what I consider a “Level 3″ technology, which I discuss below.

Pure Play Application Specific

This is where I will admit it gets a little “cloudy.” Seriously, companies such as Box.Net and EMC’s latest implementation with Mozy are appearing as SaaS storage plays and piling on the cloud wagon. I am almost certain that companies like SalesForce.com will be confused with or will legitimately become cloud plays. Probably the best definition of a “Level 1 Pure Play”  is with EnterpriseDB’s latest announcement of running its implementation of PostgreSQL on Amazon’s EC2. There are also few rumors of services that are trying to run MySQL on EC2, but most experts agree that this is a challenge on the EC2/S3 architecture. It will be interesting to see Sun’s cloud formations flow in regards to its recent acquisition of MySQL.

Pure Play Technology

When ever you hear the terms Mapreduce, Hadoop, and Google File System in regards to cloud computing, they primarily refer to “Cloud Storage” and the processing of large data sets. Cloud Storage relies on an array of virtual servers and programming techniques based on parallel computing. If things like “S(P) = P − α * (P − 1)” get you excited, then I suggest that you have a party here. Otherwise, I am not going anywhere near there. I will, however, try to take a crack at explaining MapReduce, Hadoop, and the Google Files Systems. It is no wonder that the boys at Google started all of this back in 2004 with a paper describing a programming model called Mapreduce. MapReduce is used for processing and generating large numbers of data across a number of distributed systems. In simplistic terms, MapReduce is made up of two functions: one maps Key/Value pairs, and another reduces and generates output values for the key. In the original Google paper “MapReduce:Simplified Data Processing on Large Clusters,” a simple example of using GREP to match URL’s and output URL counts is used. Those Google boys and girls have come a long way since 2004. Certainly, it is much more complicated than I have described. The real value in MapReduce is its ability to break up the code into many small distributed computations.

Next in this little historical adventure, a gentleman named Doug Cutting implemented MapReduce into the Apache Lucene project, which later evolved into the now commonly known Hadoop. Hadoop is an open source Java-based framework that implements MapRecuce using a special file system called the Hadoop Distributed File System (HDFS). The relationship between HDFS and the Google File System (GFS) is not exactly clear, but I do know that HDFS is open and that it is based on the concepts of GFS, which is proprietary and more likely very specific to Google’s voracious appetite for crunching data. The bottom line is that a technology like Hadoop and all its sub-components allows IT operations to process millions of bytes of data per day (only kidding, I couldn’t resist a quick Dr. Evil Joke here “Dr. Evil: I demand the sum… OF 1 MILLION DOLLARS “). Actually, what I meant to say quintillions of data per day.

Most of the experts with whom I have talked say that Hadoop is really only a technology that companies like Google and Yahoo can use. I found, however, a very recent blog on how a RackSpace customer is using Hadoop to offer special services to its customers by processing massive amounts of mail server logs to reduce the wall time of service analytics. Now you’re talking my language.

Level 2

Level 2 cloud providers are basically the backbone providers of the cloud providers. Amazon’s AWS Elastic Cloud Computing (EC2) and Simple Storage Service (S3) are basically the leaders in this space at this time. My definition of a “Level 2” provider is a backbone hosting service that runs virtual images in a cloud of distributed computers. This delivery model supports one to thousands of virtual images in a vast array of typically commodity-based hardware. The key differentiator of a “Level 2” provider vs. a “Level 3” is that the “Level 2″ cloud is made up of distinct single images and that they are not treated as a holistic grid like a “Level 3” providers (more on this later). If I want to implement a load balancer front end with multiple Apache servers and MySQL server on EC2, I have to provide all the nasty glue to make that happen. The only difference between running on Amazon’s EC2 and running one’s own data center is the hardware. Mind you, that is a big difference, but, even with EC2, I still might need an army of system administrators to configure file systems mounts, network configurations, and security parameters, among other things.

Amazon’s EC2 is based on XEN images, and a customer of EC2 gets to create or pick from a template list of already created XEN images. There is a really nice Firefox extension for starting and stopping images at will. Still, if you want to do fancy things like on-demand or autonomic computing type stuff, you will have to use the the AWS API’s or use a “Level 1″ provider to do it for you. I currently run this web site on an EC2 cloud. I have no idea what the hardware is and basically only know that it is physically located somewhere in Oklahoma. At least that’s what one of the SEO tools says. If I were to restart it, it might wind up in some other city – who cares? Clouds are convectious.

The biggest problem with Amazon’s EC2 is that the disk storage is volatile, which means that, if the image goes offline, all of the data that were not part of the original XEN image will be lost. For example this blog article will disappear if my image goes down. Of course, I take backups. One might say, “Hey, that is what S3 is for.” Good luck. S3 is only for the most nimbus of folks. S3 is only a web services application to put and get buckets or raw unformatted data. S3 is NOT a file system, and, even though some reasonable applications can make it look like a file system, it is still not a file system. For example, the tool Jungle Disk can be set up to mount an S3 bucket to look like a mounted file system. Under the covers, however, it is continually copying data to temporary space that looks like a mounted file system. We have found most (not all) of the open tools around S3 to be not-ready-for-production-type tools. Also, remember that EC2 and S3 are still listed as Beta applications. I list at the end of this article a number of good articles about the drawbacks of using EC2/S3 as a production RDBMS data store. Recently, an interesting point was made to me that a lot of how EC2/S3 works is really based on Amazon’s legacy. Before it offered EC2/S3 as a commercial service, it was more than likely used as its core e-tailor infrastructure. Although EC2/S3 might seem like an odd way to provide this kind of service, I am certain that it rocks as an infrastructure for selling books and CD’s.

Another player in the “Level 2” game is Mosso. Mosso is a customer of Rackspace, and it has added some secret sauce to VMWare to provide an EC2 look alike. The good news is that its storage is permanent and that there is no S3 foolishness. It will be interesting to see if Mosso can compete with a proprietary hypervisor (VMWare) vs. an open source hypervisor like XEN, which is used by EC2.

In historic IBM fashion, IBM has announced its “Blue Cloud” a few years before it plans to deliver it. Last year when it was announced, it had a specified delivery for the first half of 2008. As most of you know, I have been a flea on IBM ‘sback for over 30 years and have always been amazed at the brilliance of its slow-play marketecture strategies. It usually announces a new model for IT very early in its internal development cycle (i.e., like not started yet). Then, IBM sits back and watches the model mature and decides on the best entry point for joining. More often that not, its entry point is an acquisition. IMHO, this is a likely scenario for IBM’s entry into cloud space. The bold reality is that IBM will be a significant player in cloud computing. In fact, just its announcement last year already has put it in play. IBM’s strongest position in cloud computing is probably going to be with its enterprise facing customers. The enterprise 5k will probably trust IBM to help them make the cloud transitions when the time is right. They recently have announced a large cloud computing initiative in China. Sit back, have some popcorn, and enjoy the fun.

Obviously, other potentials are Dell, Oracle, Sun, Microsoft, and HP. There are not enough hours in a day for me to start guessing what those boys and girls are going to do. However, two giants that should be discussed are Google and Yahoo. Now, if Microsoft pulls off the acquisition of Yahoo, it will be interesting to see how or if MS can make some real commercial value out of Hadoop and other Yahoo cloud initiatives. Google is the biggest question mark. Will it ever decide to commercialize its cloud initiatives? Most experts whom I have talked to say that Google’s infrastructure is so unique that its cloud implementations probably wouldn’t make sense for anyone else but Google. Kind of like the legacy EC2/S3 story on steroids. As I have stated before, though, when the really big brick and mortar data centers start getting that deer-in-the-headlights look for IT costs, whom do you think Jame Dimon is going to call (IBM or possibly Google)? If Google gets enough of those calls, it might start to listen.

Level 3

Level 3 providers, IMHO, are the current highest level of the cloud food chain. IMO, 3Tera stands all alone at this level. I first heard about 3Tera when I read a Linux magazine article listing it as a top company to watch in 2008. 3Tera provides software that allows a company to run its own Virtual Private Data Center (VPD). A company or hosting provider can install 3Tera’s Applogic software on a grid of commodity-based hardware and enjoy all the rewards of having a self-contained cloud. 3Tera has partnered with a number of hosting providers that will provide customers with their own private VPD isolated on a grid. 3Tera is only about 3 years old, I was that that its goal was to provide Google-style commodity computing to the masses. The primary differentiator between the 3Tera offering and EC2/Mosso offerings is that 3Tera’s approach is holistic. When you get a grid using 3Tere’s Applogic software, you get a blank template, sort of like a workflow editor to build your data center. Then, you can select from a catalog of firewall servers, load balance servers, Apache servers, and MySql servers. Basically, they are predefined virtual images. The kicker, however, is that, when you select one of the cataloged servers, the Applogic software understands the context in which you are selecting the server and makes the appropriate configurations automatically. You literally drag the icons (i.e., servers) onto the canvas and then use lines to connect the servers. All of the default mounted file systems are connected in all the right ways. All the nasty network configuration parms are set up with best practices. In fact, in a demo/briefing that I had with 3Tera, it built a three-tier Apache web and MySQL grid, the complete bundle, and started the image, all in less than 10 minutes. The images were as real as any EC2 image that I have ever used. I was so blown away that I didn’t trust them at first and asked them to putty into the icons and do some basic linux system commands to make sure it wasn’t a demo system.

The original demo configuration was made up of four systems (a firewall server, load balancer, Apache server, and a MySQL server). Then, we went back and added another Apache server and a clustered MySQL server. We re-built the package and restarted, and we had six systems running. Not once did we have to touch a configuration file. Then, they started showing off by ssh’ing into the apache server and tried to ping yahoo.com. It didn’t work because their best practice out-of-the-box implementation had already isolated the servers behind the firewall. I think that at one point I shouted, “My God .. This is how IT has to be.” If you want more ranting on this, listen to my recent IT Management Guys Podcast over at Redmonk. As Cote pointed out in that podcast, it is certainly not all fairy tales and pixie dust, but it is a hell of a lot further than EC2 and S3, and look at how excited everyone is about those services. 3Tera has graciously lent me a development grid to play with, and, unbeknownst to them, I am going to see if this is something that could be used for DevCampTivoli (I guess they know now). Since I have been having so much fun on AWS, I thought that I would distribute the fun to other tools like 3Tera and start blogging about some of my experiences using its grid software.

Summary

Minus some of my possible grammatical errors, I think this is probably a good first cut on demystifying the clouds. If you have any input corrections or things that I have missed, please feel free to add comments to this blog article or contact me directly.

Disclaimer: I have not read Nick Carr’s book the “Big Switch”

Also see … Cloud Vednors A to Z

More on cloud talk…

Topics: 3tera, amazon, blue cloud, cloud computing, ec2, google, hadoop, ibm, mapreduce, mosso, nick carr, other, rackspace, redmonk, the big switch, utility cloud computing, xen | 70 Comments »

70 Responses to “Demystifying Clouds”

  1. Robert Michel Says:
    February 5th, 2008 at 12:50 pm

    I found your blog on google and read a few of your other posts. I just added you to my Google News Reader. Keep up the good work. Look forward to reading more from you in the future.

    Robert Michel

  2. John Says:
    February 5th, 2008 at 2:42 pm

    Thanks. Back at you.

  3. Tony Lucas Says:
    February 5th, 2008 at 2:51 pm

    Hi,

    There is people (ok, us!) already taking Amazon on at the Layer 0 level, although we are also offering some Level 1 features as well.

    I’d be happy to give you a full briefing on FlexiScale if you are interested.

    Also, Mosso who you mention above is not just a customer of RackSpace, they are at least partially funded by them as well.

    Q-Layer (www.qlayer.com) are also doing interesting things and should be kept an eye on.

    Tony.

  4. Matthew Small Says:
    February 5th, 2008 at 3:21 pm

    Hi John. RightScale thanks you for the good words here. Excellent article and a great breakdown. Like your other commenter, I look forward to reading more.

  5. John Says:
    February 5th, 2008 at 3:58 pm

    Tony,

    I am becoming a cloud junkie. I definitely would like to get a briefing. Contact me ad john.willis at zabovo.com

    Thanks to all for the nice words.

  6. If you think Business Service Management is “pie in the sky”, consider this — dougmcclure.net Says:
    February 5th, 2008 at 5:03 pm

    [...] What’s the future hold for Business Service Management (BSM)? Well, if John and many analysts are right it’s really going to be “in the sky” with the emergence of cloud computing. Read some of John’s thoughts on the topic in “Demystifying Clouds”. [...]

  7. Scale the Mind » Blog Archive » Cloud Computing - Demistifying Clouds Says:
    February 6th, 2008 at 1:24 am

    [...] can find the article here. tags: infrastructure | virtualization Related Posts ESX 3.5/VirtualCenter 2.5 Beta 2 [...]

  8. Bert Armijo Says:
    February 6th, 2008 at 3:14 am

    John,

    You’ve written an excellent analysis of a fast developing field. Defining characteristics for classifying competitors is no simple feat, and I’m sure it will help a lot of folks understand what’s available.

  9. John Says:
    February 6th, 2008 at 8:08 am

    Thanks

  10. Dave Durkee Says:
    February 6th, 2008 at 2:18 pm

    Hello John,
    Thank you for the insightful comments and kind words about ENKI. Our mission is increasingly becoming one of facilitating cloud computing in larger enterprises, though we’re not abandoning our startup-servicing roots. To that end, we have become a level 1/level 2 hybrid with our own boutique hosting backbone for situations where a customer can’t productively engage with larger level 2 hosting providers. The idea is to provide enterprise-grade operations services even to the smallest players, or to the smallest projects of the largest players. We see AppLogic as the best technology currently available for meeting that goal.

  11. James Urquhart Says:
    February 6th, 2008 at 2:46 pm

    I agree 3TERA is an excellent example of Level 3 as you define it, but they do not stand alone. Cassatt has been in the cloud computing infrastructure business for almost 5 years, and has several large enterprise and government customers to show for it.

    Also, I think it would be interesting to get your opinion on when one would choose to go Level 1/Level 2 vs. implementing Level 3 in their own infrastructure. My take is at http://servicelevelautomation.blogspot.com/2007/11/beating-utility-computing-lockdown-part_25.html

    Nice post, though. An interesting way to break things down.

  12. John Says:
    February 6th, 2008 at 3:11 pm

    James,

    Thanks. I will have to look at Cassatt as well. I did a quick first pass at all of the players and I missed them. Hopefully I will continue to get more feedback and I will be able to create an updated list. I look forward to reading your blog it sounds like the kind of stuff cloud junkies yearn for.

    John

  13. Rich Wellner Says:
    February 6th, 2008 at 3:47 pm

    Good article. It raised a couple interesting issues in my mind.

    The first thing I noticed was “People are always asking me (their first mistake) what is the difference between clouds and the “Grid” hype of the 1990s. My pat answer is “virtualization.” Virtualization is the secret sauce of a cloud.”

    Virtualization has been part of the grid since nearly the beginning. The project that created grid computing, Globus, has had a full scale effort going called “virtual workspaces” (http://workspace.globus.org/) for a few years now. And before that there were projects like VDT and even other projects like Condor have virtualization as important aspects of their code. My company, which is certainly a grid company, also has commercial virtualization (and service management) solutions (http://univaud.com/solutions/data-center-automation.php).

    For 11 years now the Globus project has been about creating secure common interfaces to resources for distributed computing. This includes security mapping, delegation of credentials and other essentials to make “cloud computing” a reality.

    If you look at early drafts of what IBM is now bringing to market, you’ll also see that they were called grid computing projects. The change in vernacular is very recent, and mostly marketing. Cloud computing and grid are very, very similar. If anything, grid computing is a superset of cloud computing (e.g. we’ve been doing everything that cloud computing is making noise about now for a while, but also security management (virtual organizations), distributed/hierarchical monitoring, high performance data movement, distributed data management (including application specific meta data).

    A couple things you might find interesting. Google up the 451 groups grid maturity model. You’ll see that it’s very closely related to your “levels of cloud computing”.

    Also, have a look at (http://gridgurus.typepad.com/grid_gurus/2008/01/sledding-and-th.html). This is a blog I wrote a couple weeks ago about grid and the gartner hype curve. Also very closely related to your central thesis.

    Whether we end up calling this stuff grid or cloud, it’s important to recognize the decade (or more) of work that is behind the marketing and not setup a false dichotomy between grid and cloud.

  14. John Says:
    February 6th, 2008 at 4:14 pm

    It looks like I have a bit more research to do. Your comment was excellent information and this is exactly the kind of stuff I was looking for.

    Thanks
    John

  15. Jengates Blog » Blog Archive » links for 2008-02-06 Says:
    February 6th, 2008 at 6:24 pm

    [...] Demystifying Clouds | John M Willis ESM Blog [...]

  16. Put Together Quickly » More Optimization for your site Says:
    February 6th, 2008 at 6:31 pm

    [...] out growing your current setup and need to look for something a bit more robust take a read over Demystifying Clouds by John Willis. He gives an excellent description on what cloud computing is and a run down on the [...]

  17. John Says:
    February 6th, 2008 at 10:53 pm

    James,

    Ok now, maybe, I am a little more “Educated”. Your not going to like this. I see 3Tera and Collage as apples and oranges. Collage looks like a steroids based provisioning system and maybe the only true orchestration system I have ever seen (based on your web site stuff). However, and I will admit I am probably spitting hairs, Collage does not fall into what I would call a cloud. Utiliy computing yes, cloud no. IMHO, in a cloud hardware and physical location are completely out of the picture.

    Actually most of what IBM has done so far with thier cloud initiatives is very much based on a solution similar to yours. IBM uses the acquired Think Dynamics software (now called TPM) internally to provision dev, test, and demo systems. However, nothing like you guys are doing with power management and utility services.

    In the end can Collage provide the same SLA’s as 3Tera? I am certain it can.

    johnmwillis.com

  18. John Says:
    February 6th, 2008 at 11:07 pm

    James (again),

    Your question to me is when do I think it is reasonable for a customer to go on-premise vs. off-premise.

    I think before offerings like Collage and 3Tera lines were clear. All mom and pops should go off-premise and all others should stay home.

    My experience has always been with E5k customers and it will be very interesting to see how their brick and mortar data centers react to the “utility” movement. All things being equal I think Carr is correct and computing will eventually go the way of electricity. However, enterprises like the Federal Reserve run underground hidden on-premise DR sites. There is a lot of IP/Coin out there that may never go the Carr way. However if they all do I am sure the Fed Res will be the last ones to turn out the lights .

  19. James Urquhart Says:
    February 7th, 2008 at 1:25 pm

    Thanks so much for taking the time to follow up. My responses below:

    –To your first comment–

    Hmmm. I guess I’m a little confused…

    You seem to imply that 3TERA is either a) hardware independent or b) completely hides physical infrastructure from software.

    If you mean (a), then that is easily countered by a trip to their web site. 3TERA has supported hardware platforms, just like anyone else. (x86 primarily, though the hint of SPARC coming; IDE/SATA drives; GigE).

    If you mean (b), then all I have to say is its easy to “virtualize” shared network and security infrastructure if you build your own software versions of everything. However, the hardware products that exist in these spaces are popular for a reason–they meet the performance and cost characteristics required by large enterprise systems.

    Cassatt, on the other hand, has been built from the ground up to have exactly zero effect on the execution stack of the software it manages, and near zero effect on the network and security architectures an enterprise would choose to employ. (Again, there are some hardware/OS platform choices that unfortunately have to be made, but we aim to meet the vast majority of the market requirement.) The way we manage the software stack allows applications to be migrated with ease between Cassatt environments–much like (but not exactly like) 3TERA’s story for their environments.

    That said, I won’t say 3TERA’s approach is better or worse than Cassatt’s; the purpose of my blog is not to hock Cassatt, but to open IT’s eyes to the economics and strategies that utility/cloud computing provides. In both cases, the important thing to note is that utility/cloud computing affects EVERY aspect of how IT deploys and operates software, and will likely affect many aspects of how they develop/purchase it. Choosing 3TERA won’t be any more or less painful than choosing Cassatt or IBM, at least culturally.

    The long term competition here is which approach leads to more flexibility and ease in where and how you buy compute capacity. I would argue the results of that contest are far from decided.

    If you’d like a demo of what we are doing and are in the San Jose area, please let me know. I’d be happy to continue this conversation with our platform right there to play with. Let me know: james dot urquhart at cassatt dot com.

    – To your second comment –

    Excellent. Then we agree.

  20. John Says:
    February 7th, 2008 at 2:12 pm

    I love it. This is an all time record for comments on my blog…

    c) I want the software (i.e., the cloud) to completely take the physical infrastructure out of the picture. EC2 goes a long way to doing this. With EC2 I can have a “data center” and run my little IT without ever knowing where the servers are (electricity). However, with EC2 I still have to be a sysadmin (ugh).

    With 3Tera it looks like I can have both if I am off-premise hosting if we all agree in part of Carr’s theory (I really need to read his book). The big question is how does 3Tera hold up in an in-house implementation? My guess is not to different from a well tuned utility based provisioning system. I have no experience with Cassatt however I do have a reasonable amount of experience with IBM’s TPM and I can tell you that is not a well tuned implementation out of the box. Implementing that tool at a customer site is 3x to 4x the price of software and there are only a handful of true experts that know how to implement it.

    I completely agree that the real contest has not begun yet. We are still at the jargon stage. It looks like I need a demo of your system before I go off anymore on this specific subject. I might also add that I am not in-bed with 3Tera. I just found their technology very intriguing.

  21. 5 reaons it might not be "Blue Skies" for IBM | John M Willis ESM Blog Says:
    February 7th, 2008 at 2:17 pm

    [...] Demystifying Clouds [...]

  22. Thorsten Says:
    February 8th, 2008 at 5:07 am

    John, very cool blog write-up! One prediction you could add is that every single hosting vendor will attach the word ‘cloud’ to its offering in 2008. We’ll have cloud dilution!

    I wanted to invite you to a demo of RightScale, we really provide a full application deployment solution very similar to AppLogic, but built using entirely different technology and leveraging Amazon EC2. What’s interesting with Amazon is that they’ve created an ecosystem where other players, such as us, can add value and provide a more integrated system. So while EC2 is layer 2, EC2+RightScale is layer 3.

    Keep up the fun entries!

  23. Michael Sheehan Says:
    February 11th, 2008 at 11:59 am

    John,

    Your article was a fantastic read and I will be sharing it with others in my organization. I work at ServePath, a hosting provider that didn’t “make your list” like FlexiScale, RightScale and Cassatt. As the Technology Evangelist here, I too have been struggling at the definitions of cloud, utility, grid, distributed and cluster computing as it relates to our Grid products (Grid Series and GoGrid). While technically not as in depth as your article, I wrote an article as well that attempts to define these terms as I see it, but in more of a high-level approach.
    I encourage you to take a look at our offerings and contact me should you have any questions about the products.
    This is the year of virtualized hosting, I predict, with many good offerings coming to light. It will be interesting to see your analysis a few months from now, or even at the end of the year.
    Keep up the great work!
    -Michael

  24. John Says:
    February 11th, 2008 at 12:08 pm

    I have received so much great feedback on this article I am planning on doing an updated version. To that end I am trying to talk to as many vendors as possible. I am mostly available all week and I would love to talk to you about ServePath.

    Thanks
    John

  25. Michael Sheehan Says:
    February 11th, 2008 at 12:27 pm

    Great that you are getting such good feedback. Please feel free to contact me and I can walk you through are products or get you in touch with the Grid Series/GoGrid Product Manager.

    Best,
    Michael

  26. People Over Process » IT Management Podcast #005 - The Night John Slept on a Cray Says:
    February 11th, 2008 at 5:53 pm

    [...] of our cloud-crazy talk, we jump into the cloud portion of our show. John mentions and we discuss his 0-3 level analysis of what cloud computing is. Without summarize the entire post here (just go read it!) the point is that there’s [...]

  27. links for 2008-02-14 « The Adventures of Geekgirl Says:
    February 13th, 2008 at 10:42 pm

    [...] Demystifying Clouds | John M Willis ESM Blog (tags: computing cloud) [...]

  28. Level 2 Cloud Provider Matrix | John M Willis ESM Blog Says:
    February 15th, 2008 at 11:46 am

    [...] Level 2 Providers are backbone hardware and software providers. In other words the are the primary owner of the hardware infrastructure and the cloud infrastructure software. For more on this see Demystifying Clouds [...]

  29. Look Mom, Two Nines - Amazon S3 Major Outage Today | John M Willis ESM Blog Says:
    February 15th, 2008 at 5:51 pm

    [...] Demystifying Clouds. [...]

  30. SiliconDoc Says:
    February 18th, 2008 at 1:20 am

    So let me get this straight, since I’m supposed to be demystified now….
    Cloud computing, this cloud hype, is some big giant hardware infrasructure based in some wharehouse somewhere, hooked into the internet, that sells “space and computing power” – that is also hooked with a software package that EMULATES various other well known softwares(and in a sense their hardware in their small server rooms) that are used for some businessses ?
    In other words, you’re using somebodies gigantic server wharehouse, and have this software package that some company is selling that runs on a small portion of that gigantic hardware in an EMULATION mode, and “pretends to be” or is a replacement for “your own little server and server room” ?
    Do I have the concept ?
    You’re talking about an “emu” program that pretends to be your server room, so with it, all you have to worry about is managing that great software package that runs to your little puter console you use to manipulate it, and your worries about the server stack and it crashing… aren’t your problem anymore… that problem it in “Oklahoma” or wherever else your “virtual server package” winds up running when you “reconfigure it and fire it up”… ?
    lol – Do I have the concept down ?
    This is a great way to pass the buck… no more hardware worries locally, really. If something goes down… it’s likely someone else’s fault – especially since those default best practices are “built into the virtual” (3Tera) software configuration by default…
    Ooooh, how exciting…just think of all those little server rooms that don’t need to be there anymore, and how that hardware needs to find a new home, and all those IT personnel are standing there staring at it, wondering if this or that part of it will fit in the den …:-) lol
    Very interesting.

  31. Paul Wallis Says:
    February 20th, 2008 at 9:49 am

    John,

    Here’s another comment to boost your record! I enjoyed reading your blog, this subject got quite a bit of coverage in the last couple of weeks.

    In order to discuss some of the issues surrounding The Cloud concept, I think it is important to place it in historical context, looking at the Cloud’s forerunners and the problems they encountered before being adopted.

    On my blog, http://www.KeystonesAndRivets.com, I’ve tried to do that in my “Cloud Computing” post.

    One of the current barriers in the way of The Cloud is economics. I argue that,

    “Telecom prices have fallen and bandwidth has increased, but more slowly than processing power, leaving the economics worse than in 2003″.

    And “I’m sure that advances will appear over the coming years to bring us closer, but at the moment there are too many issues and costs with network traffic and data movements to allow it to happen for all but select processor intensive applications, such as image rendering and finite modelling.”

    Comments and feedback are very welcome.

    Regards

    PJW

  32. Work: “Every cloud has a silver lining…” | Life by the park Says:
    February 21st, 2008 at 12:47 pm

    [...] здесь одна из не многих не сегодня толковых статей о cloud [...]

  33. Wolkige Semantik – Clouds und Grids at viralmythen Says:
    February 27th, 2008 at 11:45 am

    [...] dieser Rechner leitet ebenfalls nur Anfragen an ein weiteres Netzwerk – die Cloud (z.B. Amazons EC2 oder Mosso) – weiter, in der dann tatsächlich die Datenabfragen stattfinden. Im alten Netz [...]

  34. eTrobador Says:
    March 14th, 2008 at 9:32 am

    Good blog!
    I’ll see you in http://etrobada.blogspot.com
    God luck!

  35. Cloud Pontification | John M Willis ESM Blog Says:
    March 15th, 2008 at 9:09 am

    [...] I like these guys I beleive his article was way off the mark. First off as I have been staying some cloud defintions need to be draw before you talk about clouds otherwise the defintions and the diliantions can be be [...]

  36. Presently trying out « Then again, I might be wrong Says:
    March 17th, 2008 at 3:36 am

    [...] the moment I’m also trying out Tumblr. Not sure if yet another place in the cloud is a good idea, put I’m slightly seduced by the whole idea of [...]

  37. Cloud Vendors A to Z | John M Willis ESM Blog Says:
    March 20th, 2008 at 5:43 am

    [...] Also See … Demystifying Clouds [...]

  38. Matching the Customer With the Right Cloud (Part 1) | John M Willis ESM Blog Says:
    March 26th, 2008 at 6:19 am

    [...] Cloud friends (mostly vendors). I put a stake in the ground by categorizing cloud vendors as either level 1, level 2, or level 3. Although, I am still refining my definitions, the bigger question is how do I answer the simple [...]

  39. Cloud Operating System » Blog Archive » What’s In A Name? Utility vs. Cloud vs Grid Says:
    March 29th, 2008 at 7:19 pm

    [...] Simon Wardley, James Urquhart of Cassatt, James Governor at Redmonk and IBM’s Gerrit Huizenga. John Willis has sought to classify cloud computing providers. And what about grids? The IEEE has published a [...]

  40. Cloud Review | John M Willis ESM Blog Says:
    March 30th, 2008 at 6:53 am

    [...] Operating System » Blog Archive » What’s In A Name? Utility vs. Cloud vs Grid on Demystifying CloudsSobre Captchas — tikitaka.ebierzo.com on Top 10 Worst CaptchasKent on How long would it take [...]

  41. Very Good Article About Cloud Computing « The Pursuit of a Life Says:
    April 7th, 2008 at 11:59 pm

    [...] Article About Cloud Computing Published April 7, 2008 Computing Tags: Cloud Computing This article was written by John M Willis, and has a very different perspective than most of the blinkered, [...]

  42. Cloud Vendors A to Z (Revised) | John M Willis ESM Blog Says:
    April 13th, 2008 at 7:34 am

    [...] Also See … Demystifying Clouds [...]

  43. new buzzword: cloud computing at philippe::niquille Says:
    April 26th, 2008 at 2:47 am

    [...] Willis wrote a good post about the demystification of the cloud and introduces an interesting “layered” classification of cloud [...]

  44. claudio Says:
    June 4th, 2008 at 11:12 am

    awesome article helps a lot do understand same things i thought regarding to this cloud grid technology,thanks

  45. OnSaaS » Blog Archive » Defining SaaS, PaaS, IaaS, etc Says:
    June 8th, 2008 at 1:09 am

    [...] why are we defining all these terms here again when everyone else has already defined them here, here, here, here, here, here, here, here, here, etc? Heck, there’s even a definition for Web 3.0 [...]

  46. George Birbilis Says:
    June 8th, 2008 at 5:36 am

    Very nice article, thanks. I like it when someone can quickly sum up and clasify several technology offerings. Then I can follow the URLs and search for the product names mentioned much faster.

    I agree with those saying though that the cloud isn’t something new. It’s like the WWW, it’s in fact a part of the Internet, although many people believe it’s the Internet itself since it’s so popular. Similarly, the Cloud is a popular concept that can/has been implemented over the “Grid”.

  47. Tristan Black Says:
    June 12th, 2008 at 6:27 pm

    Absolutely brilliant John! You opened a lot of doors and will be referring some of discoveries on our show.
    I like your sense of humour too.

  48. Tristan Black Says:
    June 12th, 2008 at 6:28 pm

    I’m not sure what happened, but this is absolutely brilliant John!
    This is very show worthy!

  49. ElasticHosts Says:
    June 23rd, 2008 at 6:09 pm

    Hi John,

    I’d like to also mention ElasticHosts, a UK-based cloud provider.

    We support all operating systems for PC hardware, and provide a simple web interface, enabling customers to instantly scale their servers and to make snapshots of the entire running machine state for backup.

  50. From the Editor of Methods & Tools » Blog Archive » Selling Software by the Pound Says:
    June 25th, 2008 at 6:43 am

    [...] Demystifying Clouds [...]

  51. Inside the Cloud: 9 Sectors to Watch - GigaOM Says:
    July 20th, 2008 at 8:01 am

    [...] and network segments. The results are impressive: On seeing 3Tera for the first time, ESM guru John Willis was so impressed he insisted on logging in to the icons on his screen to verify that it [...]

  52. My Top 3 posts of all time | IT Management and Cloud Blog Says:
    July 24th, 2008 at 12:53 pm

    [...] Demystifying Clouds [...]

  53. 91d437f0fb56cfb5b64de81b60bd6738 » Blog Archive » Inside the Cloud: 9 Sectors to Watch Says:
    July 29th, 2008 at 12:19 am

    [...] and network segments. The results are impressive: On seeing 3Tera for the first time, ESM guru John Willis was so impressed he insisted on logging in to the icons on his screen to verify that it wasn’t [...]

  54. Pascal Charest - blog d’un consultant en logiciel libre » Blog Archive » A walk in the cloud Says:
    August 16th, 2008 at 8:13 pm

    [...] computing, as defined here, here, here, here, here and…  is still in condensation phase. Ideas appear and usability [...]

  55. iWeb Blog » Inside the Cloud: 9 Sectors to Watch Says:
    August 19th, 2008 at 9:06 am

    [...] and network segments. The results are impressive: On seeing 3Tera for the first time, ESM guru John Willis was so impressed he insisted on logging in to the icons on his screen to verify that it wasn’t [...]

  56. Solmn Says:
    August 22nd, 2008 at 11:03 am

    I am approaching the cloud from the end-user perspective, where its all foggy up there, and all that matters is how easy it is to interact with my own personal data on the smallest footprint of a device.
    I’m an anticipated fan of the CherryPal C100, which is being touted as a cloud computer. The CherryPal™ C100 desktop is about the size of a paperback book with the performance you would expect from a full-size desktop computer. It has Freescale’s triple-core mobileGT processor for multimedia performance and feature-rich user interfaces, while only consuming as much power as a clock radio. CherryPal uses 80 percent fewer components than a traditional PC, and because it has no moving parts, it operates without making a sound and will last 10 years or more. I am excited about how the CherryPal can bridge barriers to people who have not had access to computers or the internet because of money, fear, education or other challenges. I will be commenting on my experience of using it on my blog as soon as I get my own CherryPal C100! You can use CODE CPP206 to get your own CherryPal for $10 less than purchase price. CherryPal for Everyone at http://cherrypal.blogspot.com

  57. Cloud Favorites | IT Management and Cloud Blog Says:
    September 15th, 2008 at 8:25 am

    [...] Demystifying Clouds [...]

  58. Virtualization and Cloud Computing: Are they different disciplines? | CloudAve Says:
    September 29th, 2008 at 4:08 pm

    [...] To emphasize this point, I will quote John Willis [...]

  59. Clouds and Beer Says:
    December 11th, 2008 at 12:14 pm

    [...] For more details on those cloud levels, read this and [...]

  60. A Technical Manager’s Perspective » Cloud Computing Links Says:
    April 29th, 2009 at 4:15 pm

    [...] three good links that expand on cloud computing: Cloud Computing, Get you head out of the Clouds Demystifying Clouds What cloud computing really [...]

  61. Zen 2.0 » Blog Archive » Defining SaaS, PaaS, IaaS, etc Says:
    May 2nd, 2009 at 5:43 pm

    [...] why are we defining all these terms here again when everyone else has already defined them here, here, here, here, here, here, here, here, here, etc? Heck, there’s even a definition for Web 3.0 [...]

  62. Günther Gerlach » Defining Cloud Computing from the scratch Says:
    June 18th, 2009 at 1:29 pm

    [...] why are we defining all these terms here again when everyone else has already defined them here, here, here, here, here, here, here, here, here, etc? Heck, there’s even a definition for Web 3.0 and [...]

  63. John Says:
    June 18th, 2009 at 7:59 pm

    Gunther,

    Thank you for reading my blog. In my defense, this definition was made in Feb 08. As I recall, there were not a lot of definitions for cloud computing at that time. I do believe this post is outdated; however, I have not created an updated post on this subject for exactly the reasons in which you lodged your original complaint.

    Thanks
    John

  64. Raghavendra Prakash Says:
    July 3rd, 2009 at 4:37 am

    Amazing, I loved your analysis. Waiting to read more.

  65. Understanding the cloud player landscape… « RedPeak Solutions Blog Says:
    July 21st, 2009 at 8:57 am

    [...] cloud computing levels, cloud players, John Willis Back in 2008, cloud computing blogger,John Willis, posted about the different levels of cloud computing in an attempt to demystify the cloud. He [...]

  66. Saju Says:
    August 7th, 2009 at 5:28 am

    John,

    Whats in line for Level 4 ?

    Other can have a look at how clouds can help enterprises now
    http://cloud4enterprises.blogspot.com

    Saju

  67. Grid Computing und Cloud Computing: Äpfel und Birnen? - Daniel Liebhart Says:
    September 27th, 2009 at 7:37 am

    [...] unter dem Namen Cloud Computing virtualisierte Dienste anbieten. (Eine wunderbare Beschreibung von Cloud Computing durch John Willis ist in seinem Blog zu [...]

  68. Sudhir Kirloskar Says:
    December 1st, 2009 at 5:42 am

    Good analysis. I have started writing on cloud computing, to explain its concept and developments, in layman terms. Please visit http://sudhirkirloskar.blogspot.com and let me know your feedback.

  69. what about cloud computing? | all about IT infrastructure Says:
    April 22nd, 2010 at 2:56 pm

    [...] Demystifying clouds – John M. Willis [...]

  70. Is cloud computing more than just smoke? | 云生活 Says:
    April 29th, 2010 at 3:30 am

    [...] In a recent study, Forrester Research identified 11 companies that it called “cloud providers.” The list includes hosting companies like Rackspace, as well as development platform providers like Salesforce.com. Systems management expert John Willis has also created a list of cloud providers and tries to demystify the concept here. [...]

Comments