IT Management and Cloud Blog

« | Home | »

Infrastructure 2.0

By John | August 28, 2007

Last Sunday, the day after BarCampNashville, I had lunch with Luke Kanies, the owner of Reductive Labs and the author of Puppet. It became a working lunch due to all my questions about his product. I first met Luke at OSCON 2007 during his Puppet session. I hadn’t planned to attend the Puppet session because I was really trying to focus on monitoring and availability at the time since I had stepped out of the configuration management space.

Luke gave a great presentation, and I became really interested in Puppet. Back in the day, I used to work with all the Tivoli products and used to be as knowledgeable of the Tivoli configuration management tools as I was of monitoring and event correlation. Somewhere along the way, however, the Tivoli products became way too complicated for one person to be an expert with all their ESM offerings. In fact, my litmus test for a product is that, if I can’t be the smartest person in a classroom with a particular product, then I won’t teach that product. If I stop teaching the product, then I have ceased to be the smartest person with it in the room. Luke says that we should rename ESM and that he hates software. In fact, he calls himself a professional software hater (‘wareHater). In describing this distaste for software, he coined the term Infrastructure 2.0. He basically claims that he created Puppet because he was embarrassed by the current state of computer administration.The current state of configuration management products can be separated into two categories:.

Large Commercial Configuration Management Tools, which include

And Open Source Configuration Management Tools, which include

Note: I did not include IBM Tivoli Provisioning Manager and Qlusters in these lists because they are used much more as provisioning products than as configuration management tools.The deeper that I probed with my questions during our working lunch, the more I started to really like his product. In my view, Puppet differentiates itself from almost all the current products in the configuration management space in two key ways. One, Puppet tries to define all its resources in human terms (i.e., what Luke calls a cross-platform semantic abstraction).

Tivoli, to a certain extent, actually tried this approach with one of its earlier products called Tivoli User Administration (TUA). It introduced a cross platform tool for managing OS configurations. Puppet succeeds where

Tivoli fell short because Puppet not only performs the cross-platform management (i.e., Solaris, SuSe, RedHat, Debian, Centos, OpenBSD, Oracle Linux, and Ubuntu) but also allows users to define all their resources in understandable human terms. For example, Puppet allows a user to define a resource such as a file system as “filesystem” without having to know the gory details of all the file system commands and configuration files on different systems. While Puppet provides this abstraction layer for OS configurations, it also allows a user to define configurations for applications using the same abstraction layer, thereby managing the distribution and configurations of applications such as MySQL, Apache, and PostgreSQL. Imagine being able to define all of your complex software configurations and relationships in simple human terminology.

Puppet’s ability to define complex relationships in human terms brings me to the second way in which Puppet differentiates itself from other products in configuration management space: One of the Puppet project’s goals is to define a CPAN-like repository for human resource descriptions (called “recipes”) for all software products.

Here is an example of a recipe for installing and maintaining MySQL on different machines:

class mysql-server {

$password = “insert_password_here”

package { “MySQL-client”: ensure => installed }

package { “MySQL-server”: ensure => installed }

package { “MySQL-shared”: ensure => installed }

exec { “Set MySQL server root password”:

subscribe => [ Package["MySQL-server"], Package["MySQL-client"], Package["MySQL-shared"] ],

refreshonly => true,

unless => “mysqladmin -uroot -p$password status”,

path => “/bin:/usr/bin”,

command => “mysqladmin -uroot password $password”,

}

}

 

Currently, the fast-growing Puppet community has contributed about 50 recipes. Puppet supports most of the configuration management processes through “Providers,” and it comes with around 10 default providers for managing system configurations. One package provider supports most of the common Unix/Linux installers, including, but not limited to, dpkg, gem, rpm, and pkgadd on Solaris. Puppet also allows users to define other packagers by letting them create their own packages. For example, Puppet can run on AIX, but it doesn’t have “installp” support. This missing functionality could be fixed by creating a new provider. Puppet can also shell out on the agent, allowing it to install virtually any software product. In fact, I have been working with Tivoli software distribution customers for over 10 years, and I can tell you that over 80 percent of all the Tivoli software packages that I have seen customers develop have been executions of the native installs.

 Architecture
The Puppet server runs as a daemon that commutates using XMLRPC of HTTPS. Puppet uses standard SSL certificates and includes its own CA. Here is a simple example of the Puppet architecture:

puppet12.png

 

The Puppet agent wakes up every 30 minutes (this time can be configured) and contacts the server to get its current configuration. Then, the agent topologically sorts the configuration and checks the status of all the defined resources for that server. If nothing has changed for a configured resource, Puppet will take no action; it acts only when the state of the resource has changed. This feature allows Puppet to do life-cycle management for a server by doing the initial install, on-going upgrades, and de-commission. The Puppet architecture allows machines to be kept up to date and in sync with an organization’s business policies.

Puppet is written in Ruby and also provides its own language for defining resources (a language called, conveniently enough, Puppet). The Puppet language seems extremely flexible and easy to define because it comprises mostly configuration-type statements. Though it comes with preset defined functions, users can create their own Ruby-based functions as well. See the Puppet Wiki for tons of information about getting started.

Business Model

Reductive Labs, Luke’s company, provides community support for Puppet through its Wiki, mailing lists, and IRC. Though Puppet is open source GPL V2, Reductuve Labs, like any good open source vendor, provides support and services around Puppet. It currently has 5 paying customers at an average of $15k per year. It also provides a bootstrap service of on-site installation, training, and implementation. A customer can typically get the one-week implementation, training, and the first-year subscription support for around $25k. I usually try to stay way from proprietary vs. OSS price comparisons, but this cost is very insignificant compared to some of the proprietary vendors listed above. Currently, Reductive Labs is working with about 12 serious organizations that are now using Puppet among a whole slew of community users.

iLike.com

Uncomfortable with his recent celebrity at conferences, Luke told me that he has difficulty measuring his successes because he has his head so deep in the development and services of Puppet. One of his better success stories is with iLike.com, a website that allows users to download and share music. When iLike created one of the first Facebook applications, it grew from about ½ million users to over 6 million in a week. Luke, being the entrepreneur that he is, asked how iLike planned to manage that growth. He discovered that a services company in Seattle was managing iLike.com’s infrastructure build out using Puppet. In fact, one of the owners of that company told Luke that he makes a healthy living installing Puppet. Luke admitted that he felt feel pretty good to know that other people can make a living from his product.

My Cheesy Attempt at Being a Blog Interviewer

Near the end of the conversation, I felt that I needed to ask some obligatory “Blogger” type questions. First, I asked, “Why GPL?” Luke said that it’s not so much the license but what the license has afforded him to do. For example, within the last year he has been invited to give Puppet presentations (at not cost to himself) in five countries. He also feels that he could never have afforded the type of growth that he has achieved without his open source community. He also thinks that, if he had created a proprietary company, he would not have achieved even a small portion of what he has accomplished in a very short period. Also, his open source community has a lot of early adapters that thoroughly test out new functions and features, which can then be delivered earlier in the development cycle.

I then followed up with my personal “baptism of fire” question. My theory is that most successful people can pinpoint one or two “baptism of fire” moments in their career. Though at first he didn’t recall one, he eventually pointed to his time spent compiling Apache in 1998. Back when he started as a systems administrator, one had to know how to compile everything, which got him thinking about an easier way to do ESM. I also asked him about his experiences coding, and he said that he actually majored in chemistry and that the first program that he ever wrote was a 900-line shell program. Some people, I guess, are born to be coders.

I also asked him what he thought might be some future opportunities. Luke said that he saw managed services and possibly building and somehow leveraging the recipe repository. If he really succeeds in creating a sort of CPAN for configuration management, he should be able to make money on that somehow.

Summary

In all honesty, even though my focus has been on availability and event management in the OSS space, I find myself really intrigued by what Luke is doing with Puppet. In the early days of Tivoli, all the products were bound by the Tivoli Framework. At the core of the Tivoli Framework was the configuration management infrastructure. The Framework’s distribution manager made all the products, such as monitoring, inventory, user administration, and event management, viable in a large shop that had to manage many servers. Even today, as IBM/Tivoli is putting the final nails into the Framework coffin, thousands of Tivoli customers are scrambling to manage their infrastructures with all the new IBM products. As I look at all the extremely interesting OSS ESM tools out there, I struggle to understand how they will all integrate. Maybe in an “Infrastructure 2.0″ sort of way, the framework might be created by Puppet.

Topics: 451, OSS, barcampesm, barcampnashville, bmc, caos, cmdb, esm, gartner, hp, ibm, management, opensource, oscon, puppet, tivoli | 8 Comments »

8 Responses to “Infrastructure 2.0”

  1. Somewhere out there! » Blog Archive » Infrastructure Management Says:
    August 29th, 2007 at 12:12 am

    [...] on infrastructure management for Linux and Debian with puppet and other tools. Also check out this quick look at [...]

  2. People Over Process » links for 2007-08-30 Says:
    August 30th, 2007 at 2:20 am

    [...] Infrastructure 2.0 – Puppet John has a nice write-up of Puppet and his conversation with Puppet’s creator. Check out them customers too! Congrats to Luke! (tags: puppet changemanagement configuration itmanagement gpl opensource) [...]

  3. adam Says:
    August 31st, 2007 at 12:04 pm

    The services company that helps iLike is HJK Solutions (http://www.hjksolutions.com.) Puppet is central to our efforts to help our clients build scalable, robust, easy to maintain systems architectures. Luke has built a great tool, and it’s future only looks brighter.

  4. Puppet, iLike and Infrastructure 2.0 Says:
    August 31st, 2007 at 12:31 pm

    [...] the founders of Gulf Breeze Software (an IBM Tivoli consulting house,) met up with Luke Kanies and interviewed him about [...]

  5. What if I only had a hour to live? at John M Willis Says:
    December 6th, 2007 at 8:18 am

    [...] and OpenNMS in record breaking time (about eight minutes). I also throw in a plug for what I call Infrastructure 2.0 and Puppet (Luke rocks) and they are happy and I am off of the phone with only 5 minutes to live. [...]

  6. Nicholas Carr Might be Right at John M Willis Says:
    December 14th, 2007 at 8:27 am

    [...] Infrastructure 2.0 [...]

  7. My Top 17 Favorite Posts of 2007 at John M Willis Says:
    December 15th, 2007 at 7:30 am

    [...] Infrastructure 2.0 [...]

  8. Cloud Cafe #18 - The Puppet vs. The Meat Cloud | IT Management and Cloud Blog Says:
    October 26th, 2008 at 4:13 am

    [...] Infrastructure 2.0 [...]

Comments