Unattended chef client installation

chef

Has anyone tried performing an unattended chef client install – installing chef client from the node itself?

There is not much information provided in the documentation. Also, I don't want chef-solo but chef-client on the node.

Best Answer

It isn't clear from your question which OS you are doing this on and if you are doing it in a stand-alone or chef server environment.

Here are two options for stand-alone install of chef-client:

Omnibus installer

curl -L https://www.chef.io/chef/install.sh | sudo bash

or for a specific version

curl -L https://www.chef.io/chef/install.sh | sudo bash -s -- -v 12.0.2

Package install

Example for Ubuntu:

wget https://opscode-omnibus-packages.s3.amazonaws.com/debian/6/x86_64/chef_12.5.1-1_amd64.deb
sudo dpkg -i chef_12.5.1-1_amd64.deb

Bootstrap

Knife bootstrap will install the chef-client on a node in a chef server environment. Bootstrap uses the omnibus installer and then copies the required client configuration from the server.

knife bootstrap 123.45.6.789 -x username -P password --sudo

Unattended Bootstrap

The chef-client can be installed using an unattended bootstrap. This allows the chef-client to be installed from itself, without using SSH. For example, machines are often created using environments like AWS Auto Scaling, AWS CloudFormation, Rackspace Auto Scale, and PXE. In this scenario, using tooling for attended, single-machine installs like knife bootstrap or knife CLOUD_PLUGIN create is not practical because the machines are created automatically and someone cannot always be on-hand to initiate the bootstrap process.

When the chef-client is installed using an unattended bootstrap, remember that the chef-client:

Must be able to authenticate to the Chef server
Must be able to configure a run-list
May require custom attributes, depending on the cookbooks that are being used
Must be able to access the chef-validator.pem so that it may create a new identity on the Chef server
Must have a unique node name; the chef-client will use the FQDN for the host system by default

When the chef-client is installed using an unattended bootstrap, it is typically built into an image that starts the chef-client on boot. The type of image used depends on the platform on which the unattended bootstrap will take place.

Use settings in the client.rb file—chef_server_url, http_proxy, and so on—to ensure that configuration details are built into the unattended bootstrap process.

A node’s initial run-list is specified using a JSON file on the host system. When running the chef-client as an executable, use the -j option to tell the chef-client which JSON file to use. For example:

chef-client -j /etc/chef/file.json --environment _default

where file.json is similar to:

{
  "resolver": {
    "nameservers": [ "10.0.0.1" ],
    "search":"int.example.com"
  },
  "run_list": [ "recipe[resolver]" ]
}

You may also want to check out Provision and Bootstrap AWS instances with Chef

Related Solutions

How to get started with Chef

Updates

Chef version is 0.10.8
The Chef Wiki has gone through a lot of revision since this was originally asked.
The Opscode Platform is now called Opscode Hosted Chef
Chef will soon be distributed as a full-stack installation, in addition to RubyGems and Debian packages.
Vagrant is a great way to get started using Chef.

Original response follows

First, as of this writing, the latest version of Chef is 0.9.8. Opscode distributes Chef as RubyGems, and has up-to-date Debian/Ubuntu packages as well. RPMs for the latest version are released by a member of the community.

Second, in versions 0.8.x+ and higher, OpenID is no longer required for authentication. Merb is required for the Server because it is a Merb web application.

Opscode hosts a Chef Server as a Service called The Opscode Platform. You can signup and get started. It is free during the "feature beta", and once it leaves beta it is free for up to 5 nodes. If you sign up there's a getting started guide on Opscode's help site.

Overview of getting started:

Get Chef installed. The easiest way to do this is to run an Ubuntu 10.04 system, either in a VM or on EC2, etc. The first step is to get Ruby installed, then install Chef itself.
Set up the local user environment to communicate with the Opscode Platform, or your own Chef Server. You can set up your own server by using Opscode's Chef bootstrap configuration for RubyGems.
Set up a Chef Repository to store your cookbooks and other infrastructure configuration.
Download cookbooks you want to use into the 'cookbooks' directory in the repository.
Upload cookbooks to the Chef Server, add the recipes to the run list of a node and watch it get configured.

As for Django and Apache2:

You'll need to write one that deploys your Django application code. The Apache2 cookbook contains a Definition to help set up a virtual host for web applications.

You can help improve the documentation by submitting a ticket in the "Documentation" component of the Chef project. If you have trouble with the Open Source Chef Server, the best places to ask for help are the Chef Mailing List, or the #chef IRC channel on FreeNode. If you sign up for the Platform you have access to the help site to submit support requests.

Disclosure: I work for Opscode.

Chef best Practices/Questions

Edit This question and answer are years old. The definitive best practices are taught via the Learn Chef Rally self-paced training modules produced by Chef Software, Inc. The bulk of the original answer is below.

In this answer, "Chef" or "chef-client" usually refers to Chef Infra, the product. Opscode renamed to Chef Software, Inc in 2013. In April, 2019, Chef opened the source code for all its products, along with creating consistent brand naming.

Not clear if it's better to setup roles in ruby DSL, JSON, or from the management console? Why are there multiple ways to do the same thing?

2019 Update: Policyfiles are the best workflow to use. Roles are considered an inferior practice, and Chef Software, Inc. recommends migrating to Policyfiles.

There are multiple ways to do the same thing because people have different workflows. You pick the workflow that is best for your environment. Let me explain what the differences are so you can make an informed decision.

The Ruby DSL for Roles exists to make it easier to write roles without knowing the syntax of JSON. It is a simple way to get started with Roles. Once you have made changes, you upload them to the Chef Server with knife.

knife role from file myrole.rb

This converts the role to JSON and stores it on the server. If you have an environment that enforces the Chef Repository where your roles live as the source of truth, this works quite well.

JSON is what the Chef Server stores, so you also edit JSON directly in the management console. It does require more fields than the Ruby DSL in order for Knife to recognize it properly to upload. Those details are hidden to some degree via the web UI.

The disadvantage of using the webui/management console for editing roles is they aren't in your local version control system unless you download them from the server. You can do this with knife:

knife role show myrole -Fj

The -Fj tells knife to "display in JSON format." You can redirect the output to a .json file if you like.

Years ago update: There are additional knife commands for working with the files in the local chef repository. Currently these commands support only JSON format files. A community RFC is open which will address adding support for the Ruby DSL for these plugins. Here's a brief summary of the workflow.

Check content differences between the server and the local file.

knife diff roles/myrole.json

Upload a JSON formatted role file. The roles/ path is required. This gets mapped to the same API endpoint on the server.

knife upload roles/myrole.json

Download the content from the server overwriting the content of the file in the repository.

knife download roles/myrole.json

These commands come from knife-essentials, which is built into the chef client package.

Can you organize cookbooks into subdirectories? eg- we have custom software that I'd like to write a cookbook for and stick that into: chef-repo/cookbooks/ourcompanystuff/customsoftwarecookbook would this be a good practice?

No. Knife has an expectation of where cookbooks should live because it uses an API to upload cookbooks to the Server. This is set in the knife.rb with cookbook_path. In older versions of Chef Infra, you could specify an array of paths for cookbooks, but this is being deprecated because it required more maintenance and was confusing to users.

By convention we name customer specific or site specific cookbooks with the name prefixed in the cookbook diretory. For your example, it would be:

chef-repo/cookbooks/ourcompany_customsoftware

There might be multiple different cookbooks for "ourcompany" depending on what you're doing.

Further reference:

Do I create a cookbook for each type of role that specifies what it does? Do I have these cookbooks include other cookbooks (i.e.- the cookbook for my webserver role includes the apache cookbook). I'm not sure how cookbook inter-dependencies and inheritance are handled.

There is no direct relationship or dependency between roles and cookbooks.

Roles have a run list, which specifies the recipes and other roles that should be applied to any node that has that role. Nodes have a run list that can contain roles or recipes. When Chef runs on the node, it will expand the run list for all the roles and recipes it includes, and then download the cookbooks required. In a node run list:

recipe[apache2]

Chef will download the apache2 cookbook for the node so it can apply this recipe.

You might have a cookbook specific for a role in your infrastructure. More commonly you'll have cookbooks that are for setting up certain types of services like apache2, mysql, redis, haproxy, etc. Then you would put those into appropriate roles. If you have custom application specific things that need to happen to fulfill a role, then you could write this into a custom cookbook (like I referenced above).

Further reference:

Is there anything like puppets external node classifier so nodes automatically determine their roles?

"Yes." The Chef Infra Server does node data storage (in JSON) automatically, and the server also automatically indexes all the node data for search.

Further reference:

It seems like you can configure things with knife or within the management console, or editing JSON files? This is super confusing to me why there are so many ways to do things, it's paralyzing! Is there a reason to use one or the other?

The Chef Infra Server has a RESTful API that sends and receives JSON responses. Knife and the management console are user interfaces for interacting with the API from an administration point of view.

You can use the tool you like better, though the management console doesn't have as many features as Knife. Most people that use Chef Infra prefer the command-line interface for the power and flexibility it provides, even folks who are using Chef Infra on Windows. Further, knife is a plugin based tool that you can create new plugins to interact with the Chef Infra Server, or with other parts of your infrastruture.

Chef Infra is a set of libraries, primitives, and an API. It gives you the flexibility to build the configuration management system that works best for your infrastructure.

Best Answer

Omnibus installer

Package install

Bootstrap

Unattended Bootstrap

Related Solutions

How to get started with Chef

Chef best Practices/Questions

Related Topic