Cooking your own configuration in the cloud

Although configuration is one of the most important aspects of cloud deployment, it is often neglected. There are several automation tools and platforms available, such as Opscode Chef or Puppet, but a minimal infrastructure is usually required to be able to use them. They are really good at automating configuration in large environments, but this is not always the use case in the cloud.

People want to create services in the cloud quickly and easily, without having to worry about infrastructure apart from the services that they are deploying. So how could service configuration be automated? SSH or command-line scripts are often very complex to build and maintain, and making them work on multiple operating systems can be painful. How could this be resolved quickly and easily? The answer is to team up Jclouds and Chef Solo. They’re a perfect match!

Cool features of Chef Solo:

  • Run cookbooks on your nodes without a Chef Server.
  • Many cookbooks are available, compatible with most common operating systems.
  • Configure roles and data bags.
  • Schedule Chef runs to keep your node up to date.

Ok, this sounds great, so let’s get into the code!

Cooking a basic web server

This first basic example will install an Apache2 web server on a node. It will download a tarball with all the cookbook definitions and install the Apache2 recipe on the deployed node.

// Create the connection to the cloud provider
ComputeServiceContext computeContext = ContextBuilder.newBuilder("<provider>") 
    .endpoint("<provider endpoint>")
    .credentials("<identity>", "<credential>")
    .modules(ImmutableSet.<Module> of(new SshjSshClientModule()))
    .buildView(ComputeServiceContext.class);
ComputeService compute = computeContext.getComputeService();

// Build the Chef Solo script that will download the cookbook definitions
// and install the selected recipes
Statement bootstrap = ChefSolo.builder().
    .cookbooksArchiveLocation("http://foo/bar/cookbooks.tar.gz")
    .runlist(RunList.builder().recipe("apache2").build())
    .build();

// Select the template to be deployed
Template template = compute.templateBuilder()
    .imageNameMatches("Ubuntu.*")
    .options(runScript(bootstrap))
    .build();

// Deploy the node and bootstrap it using Chef Solo!
compute.createNodesInGroup(group, 1, template)

Have you noticed that there are only 5 lines of code? That’s how easy it is to bootstrap nodes with jclouds and Chef Solo!

Playing with roles and data bags

Chef Solo also lets you define roles and data bags on the fly, to provide more flexibility to your configuration. This example will show how a role can be created and used to customize the Apache2 installation.

Role role = Role.builder()
    .name("webserver")
    .description("Web server with apache and SSL")
    .jsonDefaultAttributes("{"apache": {"listen_ports":["8888"]}}")
    .runlist(RunList.builder().recipe("apache2").build())
    .build();

Statement bootstrap = ChefSolo.builder().
    .defineRole(role)
    .cookbooksArchiveLocation("http://foo/bar/cookbooks.tar.gz")
    .runlist(RunList.builder().role("webserver").build())
    .build();

That’s it! With just these two lines you can create a role to set up the default configuration for your web server deployment.

Automating maintenance

In the examples I’ve used the cookbooksArchiveLocation to download a cookbook archive. This could also be a local file system path if you are able to upload the cookbooks to the node (for example, using the jclouds SshClient).

Instead of downloading or uploading a cookbook archive to the node, another common use case is to clone a git repository with all the cookbooks and set up a cron job to keep it up to date. Then the Chef Solo statement can be configured to run periodically, and this way the node will always be up to date with the latest version of the cookbooks.

// Clone all community cookbooks in the '/var/chef' directory
Statement cloneCookbooks = CloneGitRepo.builder() //
    .repository("git://github.com/opscode/cookbooks.git") //
    .directory("/var/chef") //
    .build();

// Configure Chef Solo to read the cookbooks from there
Statement chefSolo = ChefSolo.builder().
    .cookbookPath("/var/chef/cookbooks")
    .runlist(RunList.builder().recipe("apache2").build())
    .build();

// Build a boostrap statement that installs Git, clones the cookbooks and
// runs Chef Solo
StatementList bootstrap = new StatementList(
    new InstallGit(), cloneCookbooks, chefSolo);

As you’ve seen here, bringing configuration management to the cloud is easy with jclouds and Chef Solo. There is no need to build complex infrastructures or deployments. Just start cooking your own stuff!

When we finished our Chef integration, we wanted to give our users a template that was ready to use with Abiquo. Creating the template was a bit more painful than I expected at the start.

ntp

One thing we omitted in the first version was the time sync. If the difference between the clock of the machine trying to access the Chef server and the Chef server itself is more than 15 minutes, the server denies access with a 401 error.

This means our template needs to be synchronized, so we made the following changes:

  • Update all hypervisor clocks. A virtual machine gets the time from the hypervisors when it boots. If all your infrastructure is already in sync with ntp, for example, this will be easier.
  • Force synchronization when the VM boots up. We added a few lines to the init script to stop the ntp server, run the ntp client and start the ntp server again. Remember that ntp only makes small corrections to the time. If your VM is getting a date and time that is too far from the right one, ntp will not be able to sync the time in the first run and it will cause Chef to fail on the first attempt. Sync your hypervisors!

OS

Our initial idea was to release 3 templates with the Chef agent: CentOS 5.7, CentOS 6.2 and Ubuntu 11.04. For testing, we used the Opscode community repository. We started testing with the WordPress recipe because it has useful dependencies like MySQL and Apache. But the tests failed. When we looked into it, we found the recipe was trying to install packages that do not exist in certain distributions (CentOS 5.7 and CentOS 6.2).

We fixed a few things but we were unable to believe that anyone would be using these recipes as they were. We asked on chat and found out that most of the recipes are broken for all other operating systems except Ubuntu. So we moved to Ubuntu and everything started working better.

The only fix we made was adding an ‘apt-get update’ because sometimes we got an error that some packages could not be found because the DB was outdated.

Recipes

After we got WordPress working we wanted to get the Jira recipe working. Jira is a program by Atlassian that is used to manage tickets. It’s a really powerful bug tracker that we use in the development of Abiquo.

The first approach was something similar to the flow presented in this video but instead of getting a screen from Jira, we got an error screen from Apache. It appeared that Tomcat was not starting up.

We started debugging and we found that the recipe requires manual steps to configure the database. After deploying your recipe, you need to log in to the VM and run some commands and after that you can run Jira. Honestly, I was a bit surprised by that. It is true that the recipe automates most of the installation, but why force the user to run commands to create the DB and configure a user to connect to the database?

The answer was in the readme. This recipe was written some time ago, when the MySQL recipe was not yet written. There is a note in the readme explaining that the manual steps are there until someone writes the MySQL recipe. But no one had updated the Jira recipe.

In the end, we updated the recipe to use the MySQL server recipe and to use the new version of Jira (5.0.2).

We will provide a git repository with these fixes. And we have made a pull request to the Opscode repository in order to contribute this recipe upgrade.

Demonstration of how you can use Abiquo to deploy a Jira server for your team in just a few minutes. Download the Chef template, select the resources you want to use (check the hardware requirements), select the Jira recipe and deploy. Of course, check that your VM has a usable IP address.

http://www.youtube.com/watch?v=pA6ZL3T6NZ4

 

P.S: The official Jira recipe doesn’t work. I submitted a pull request but it is still pending to be applied. Until the code is merged to the official repository, you can use my fork.

https://github.com/Marc-Morata-Fite/jira/tree/jira-update-to-5