Manage chef cookbooks in a team environment

chef

I'm learning chef and having problems structuring everything to work with my team.

For starters, it seems that you should create a chef-repo folder, where you will store and modify the cookbooks used to manage your nodes.

I work on various projects, and each one of them is under git source control already. Ideally I would keep one chef-repo folder in each one of my projects with that projects cookbooks right?

However, in the chef-repo folder I have to add a configuration folder (.chef) with my knife configuration and my key validations, and these are specific to me. Is it normal to just add the .chef folder to the gitignore file?

I understand the cookbooks get uploaded to the chef server to then be deployed. How do other teams separate the staging from production environments without duplicating a lot of work? We have a master branch which is our production branch, a dev branch which is our staging branch (receives less than 5% of the website's requests) and feature branches. Most of the time, the dev branch when stable is merged to the master branch. How can we upload the cookbooks separately to be able to have two environments in a separate way?

Thanks for the help!

Best Answer

I work with multiple projects, so cjc's solution won't work for me. There's also an issue of common vs custom configuration (addresses etc are common to the company, there's also a bit of magic in the configs). The scheme I finally settled on is a bit of a hack, but it's a convenient-to-use one.

Instead of global ~/.chef, I use '.chef' subdirectory within chef-repo, which is not stored in git (it's added to .gitignore). I have also file config/knife.rb file which is checked into Git and contains shared configuration. It starts with this snippet:

root_dir = File.join(File.dirname(__FILE__), '..')
%w(knife-secrets.rb knife-local.rb).each do |conf_name|
  conf = File.join(root_dir, ".chef", conf_name)
  Kernel::load(conf) if File.exists? conf
end

This loads files .chef/knife-local.rb containing custom configuration (in basic version it's just OPSCODE_USER='username' constant that is used later on, but it can contain any knife config), and .chef/knife-secrets.rb which contains shared secrets (AWS keys etc).

Below that, there is regular knife config that uses constants defined in these files, e.g.:

client_key               "#{root_dir}/.chef/#{OPSCODE_USER}.pem"

This way, I achieve standarization of knife config across the company, which in turn means that any code snippet or knife invocation shared in a wiki will work for everyone. There is enough confusion and magic in knife itself - different configurations would only make it worse. Also, everyone gets benefit of small magic snippets, like this one to make knife ssh use login configured in user's ~/.ssh/config

There's also issue of shared secrets: chef server's validation key, AWS keys stored in knife-secrets.rb, EC2's SSH private key, encrypted data bag keys, and so on. We definitely don't want them to be stored in the repository - or, actually, anywhere where they aren't safely encrypted. So we distribute those files as a .tar.gz file, which is GPG-encrypted to everyone in the company, and shared over Dropbox.

Configuring all this is getting complicated, and I want people on team to actually use the thing, so there's the final element: rake init task which creates .chef directory, symlinks config/knife.rb there, decrypts and untars chef-secrets.tgz file, makes sure user's private Opscode Platform key is there and .chef/knife-local.rb is properly configured, symlinks knife plugins, and sets proper permissions on the directory and files inside. This task is set up so that it's safe to run it many times on already initialized repository (e.g. to update secrets or knife plugins).

There's also a helper task that repacks all the secrets, encrypts the tarball to everyone, and copies it into dropbox, to make it easier to add new employees or change secrets.

Regarding multiple environments: Chef has a feature called environments. I haven't used it yet, but it should do what you need. You can also strictly separate production environment (to avoid developers having any keys that relate in any way to production env) by having two separate Hosted Chef organizations or Chef servers. This knife.rb snippet show how to configure knife in a different way based on currently checked out branch - you can use this to set environment as well as chef server's url. There's also knife plugin called knife-flow , providing more complete, two-organization workflow.

Related Topic