When it comes to managing infrastructure deployments of a certain size, even a ninja sysadmin can't do it all without some help. In fact, one might argue that knowing when and how to use IT automation tools for managing infrastructure is a prerequisite to getting your admin blackbelt. Enter Puppet.
Puppet is an open source framework for templating repetitive tasks to save time and make them easy to document, modify, and replicate. And it's backed by a company that provides enterprise support while still conducting their development out in the open.
One area where Puppet has seen strong adoption is with deploying OpenStack. I spoke with Chris Hoge, Senior Software Engineer for OpenStack at Puppet Labs, to learn a bit about Puppet and how it can be used to automate processes for OpenStack installation and management.
How did you get started working on OpenStack and Puppet?
I came from the University of Oregon where I administered a research OpenStack installation, and it was deployed using Puppet. That’s where I got my start with OpenStack back in the Essex [release] days. I was admin of the production Folsom installation, then I upgraded to Grizzly. And when the funding for that project ran out at the university, it was a great opportunity for me to come and continue the network at Puppet Labs. I’ve been here since June of last year, and I’ve been taking over the efforts of the Puppet OpenStack modules and the community of developers which has grown up around these modules.
Tell us a little bit about Puppet as a software tool.
So, Puppet is a tool that is used for the configuration of anything. It’s a part of a movement that we would call DevOps. You specify what you want an instance to look like in a declarative language that lets you specify in as platform-independent way as possible what you want this instance to look like.
How does Puppet Labs work with the open source community?
Puppet Labs follows the proven model of doing development on our products in the open with the open source community. So all of our major products—Puppet, MCollective, Hiera, Facter—all of these things are developed out in the open, and we produce open source versions of these. This is a way for us to have a lot of visibility in the community and have a lot of feedback as to how the projects should grow and what direction they should go in. What Puppet Enterprise does is it takes all of these open source projects and bundles them together in a way that makes it easier to use the projects together, but it also gives you a support channel so that you can have large, institutional deployments of Puppet and all of its related tools, and be sure that you’re going to have professional and timely support.
How does Puppet help with OpenStack deployment and management?
Where the Puppet OpenStack modules really excel is in the deployment of OpenStack. This is a project that has been going on for quite some time now. We’ve been through several releases. A survey was conducted at the previous OpenStack Summit in Hong Kong, and it turned out that the Puppet OpenStack modules were the number one way that people were using to install and manage their OpenStack installation. This is because there are a lot really fantastic tools out there that are building on the modules. If you look at work that groups like Red Hat and Cisco are doing, where they’re taking the modules and extending them. There are a number of really great things that the modules do. They actually allow you to manage your infrastructure.
A few examples of this are being able to manage Keystone, where tenants, roles, and users can all be managed within Puppet code. We have custom pipes and providers that allow you to treat users and tenants as resources. This is also true for Neutron, where you have these abstractions of networks and subnets and ports and all of these can be managed with Puppet resources so that when you bring your OpenStack installation up you can already have your public and private networks available to your tenants completely defined. And not only during installation. You can also add and create new ones and destroy those networks as you need. And so it ends up being a really powerful tool for integrating with things like software defined networking (SDN), which is really interesting to me.
Tell me a little bit more about the Software Defined Networking modules.
That’s been one of the really exciting things that’s been going on with Neutron development and the ML2 framework where you’re taking the idea of networking and you’re trying to abstract away "how am I implementing my network"—is that through hardware, is that through virtual software switches?—and giving you an API layer to interact with that. And so it’s really exciting when we’re able to build types of providers on top of the interface that Neutron is already giving to us. When you have your network that is backed by whatever hardware vendor you’re using or whatever software implementation that you’re using behind that, you can define what you want your networks to look like. You can define the networks and the subnets, and have that be available and be managed through a central location, in a fairly straightforward declarative language.
What are some of the other modules you have which work with OpenStack? Do you have one for all of the major OpenStack components?
Effectively, we track every project that goes into the core OpenStack distribution, so of course that means that we’re going to have modules for Keystone, Nova, Glance, Swift, Neutron—your primary OpenStack projects. We also have modules that target Heat, Ceilometer, Tempest—these newer projects that you’re seeing being pulled into the OpenStack ecosystem. Essentially what we’re trying to do is that every time that a project goes from incubated status into [the core of] OpenStack, we’d like to have a module in place that targets the installation and management of that service.
How does your release schedule line up with the release schedule of OpenStack itself?
In the past, we’ve generally lagged a few months behind the release of the core OpenStack project. One of the things that I’ve been tasked with, and that I’ve been working fairly hard on, is accelerating that release cycle. This means improving our own testing structure internally here so that Puppet Labs, who curates the release of these modules, can increase our cadence on matching the release of the modules up with the release of OpenStack. We’ve taken ourselves from where it has taken weeks to validate these modules to where I can validate a module with the public repository over the course of a few hours. And so that’s actually what I’m working on right now, so that I can have a full release of the stable Puppet OpenStack Icehouse modules ready by the time we all arrive at the Summit.
It sounds like Puppet is going to be an active part of OpenStack Summit next week?
Yes. There are a number of things that we’re going to be doing there. The first is that we have a booth there, as we have for several Summits now. We’re also going to be doing a fifteen minute theater demonstration. But there also are a couple of exciting things that are going to be new this year that are targeted towards operators and developers. The first is that there’s going to be an Ops Summit, and Puppet is going to have a presence at that on Friday, May 16, at 3pm.
There was also a program that we piloted last year that was a Puppet OpenStack Design Summit meeting specifically for Puppet OpenStack. This is something that we just put together on our own, and we had some tremendous attendance to it. A lot of developers from all over wound up coming to this. We set our plans for the next year and really treated it like a traditional OpenStack Design Summit. This turned into a larger program, where a lot of open source projects who are related to OpenStack were able to have space and time set aside by the foundation so that could meet and plan their own projects. The Puppet OpenStack developer session is going to be meeting on Tuesday, May 13, from 2:50 - 6:10pm.
Outside of getting the modules updated for Icehouse, what other new features are you working on?
New Sahara and Trove modules are something that we’re very interested in working towards. I’m also personally interested in expanding our types and providers, so we can capture more of the resources that we can manage in your OpenStack cluster, particularly being able to provision machines from Nova directly from Puppet code. There’s also a huge amount of interest within the community for providing high availability for OpenStack, and that’s something that’s going to be one of my top priorities for the Juno cycle.
Is your community participating in the Puppet OpenStack project?
Absolutely! In fact, this is one of those things where we get a tremendous amount of code from outside of Puppet Labs. There are so many different organizations which are using the modules in production, and we are keeping track of where it is going, and so we get almost all of our contributions from outside of Puppet Labs actually. Which for me is another thing that is really exciting about this. The Puppet OpenStack modules are really and truly a widely distributed open source project.
That’s really the magic bullet, isn’t it? Get something started with a cool idea and your community will practically write it for you?
Yeah, absolutely! I can’t tell you how many times I’ve said to myself, "I need to implement this new feature," and then it feels like literally within minutes of thinking that to myself, a pull request comes in for it. It’s really fun to see that.
Where can we learn more?
I’d really encourage anybody who is interested in the modules to take a look at what we’re doing, whether that be from an operator’s point of view or a developer’s point of view. We have a new Puppet Labs OpenStack module that’s going to be coming out for the Icehouse release, which is going to give people a really good start for getting OpenStack installed but also give them an understanding of how they can use and extend the modules for their own environment.
We’re developing all of the source code out there on StackForge, and so people are welcome to come, join, and contribute. We also have a Puppet OpenStack mailing list that they can become involved with, and get updates on where development is going. We also have a Puppet OpenStack IRC channel (#puppet-openstack on Freenode) if they wanted to stop by with any immediately questions.