Chef - solo (chef-solo and knife-solo)

summary from book ‘Cooking infrastructure by Chef’ by Alexey Vasiliev and official chef documentation.

• CHEF SOLO
  • VENDOR COOKBOOKS
  • NODES
  • ATTRIBUTES
    • attribute precedence
    • attribute types
  • ROLES
  • ENVIRONMENTS
  • DATA BAGS
• SITE COOKBOOK
  • ATTRIBUTE FILES
  • RECIPES
    • include other recipes
    • write to log from within a recipe
  • RESOURCES
    • popular platform resources
    • custom resource files
  • TEMPLATES
    • variables inside template file
    • file specificity
    • partials
  • METADATA.RB

participants:

1. chef node (deployment target - this is what configured, has chef-client or chef-solo installed)
2. chef server (combined with chef node when using chef-solo)
3. workstation (from which knife or knife-solo is run - can be remote host as well)

core principles:

• idempotence

• thick clients, thin server

  chef does as much work as possible on the node and as little as possible on the server

• order matters

  within a recipe resources are applied in the order they are listed

chef-client

• installed on each node managed by chef server
• used to bring the node into the expected state

CHEF SOLO

chef-solo

• open-source version of chef-client with limited functionality
• doesn’t require access to chef server
• requires that a cookbook be on the same physical disk as the node
• doesn’t support authentication and authorization

knife

• provides interface between local chef-repo and chef server

knife-solo

$ gem install knife-solo

• installs knife as dependency
• adds 5 subcommands to knife:
  • knife solo init (create a new kitchen)
  • knife solo prepare (install chef-solo on target host)
  • knife solo cook (upload current kitchen to target host and run chef-solo there)
  • knife solo bootstrap (prepare + cook)
  • knife solo clean (remove uploaded kitchen from target host)

berkshelf

• manages cookbooks and their dependencies (like bundler for rubygems)
• librarian-chef - alternative

kitchen

• chef-repo
• located on workstation
• uploaded to chef server (or chef node when using chef-solo)

create a new kitchen in current directory:

$ knife solo init .

file structure:

.chef/knife.rb

• chef-repo specific knife settings (primarily paths to cookbooks, node files, etc.)
• loaded every time knife is run

VENDOR COOKBOOKS

Berksfile:

cookbook 'apache2'

install cookbook globally into ~/.berkshelf/cookbooks/:

$ berks install

install cookbook both locally into cookbooks/ and globally into ~/.bershelf/cookbooks/:

$ berks vendor cookbooks

NOTE: command berks install --path cookbooks does the same but is deprecated.

it’s necessary to specify directory (cookbooks/) - otherwise berkshelf will install cookbook into berks-cookbooks/.

knife[:berkshelf_path] option has no effect on where berkshelf installs cookbooks

• most likely it’s used by knife to search for cookbooks while provisioning. but since berks vendor command installs cookbook into ~/.bershelf/cookbooks/ as well I can’t figure out how you might end up having cookbook installed locally only (which would justify existence of knife[:berkshelf_path] option).

NODES

NOTE: nodes are configured in node files for chef-solo only - chef-client interacts with chef server to retrieve node configuration.

node

• contains node-specific attributes and run-list
• represents machine (physical, virtual, etc.)
• separate JSON file for each node in nodes/
• node file name (as a rule): DOMAIN.json

run_list

• main key in node file
• contains array of recipes and roles
• recipe name format (2 variants):
  • COOKBOOK::RECIPE
  • recipe[COOKBOOK::RECIPE]
• default.rb - default recipe of cookbook
  • executed when only cookbook name is specified without specific recipe
  • can be explicitly called with COOKBOOK::default

sample node file:

{run_list: ['recipe[apache2]']}

ATTRIBUTES

attribute

• specific detail about node
• provided from the following locations:
  • node files
  • attribute files (in cookbook)
  • recipes (in cookbook)
  • roles
  • environments

accessor methods are automatically defined for attributes in Ruby files (that is it’s possible to use dot syntax instead of hash keys):

default['apache']['dir'] = '/etc/apache2'
default.apache.dir = '/etc/apache2'

or else it’s possible to use symbols instead of strings as hash keys:

default['apache']['dir'] = '/etc/apache2'
default[:apache][:dir] = '/etc/apache2'

attribute precedence

in general attribute precedence is organized in such a way as:

• to enable cookbooks and roles to define attribute defaults
• for normal attributes to define the values that should be specific for a node
• and for override attributes to force a certain value, even when a node already has that value specified

attribute types

see also http://stackoverflow.com/a/20835500

• default
• force_default (cannot be set in node file)
• normal
• override
• force_override (cannot be set in node file)
• automatic (cannot be set manually, collected by Ohai)

attribute type is specified when using attributes by calling appropriate function of node object:

# default
node.default['apache']['dir'] = '/etc/apache2'

# override
node.override['apache']['dir'] = '/etc/apache2'

NOTE: cookbook attribute files are evaluated in the context of node object - that is why it’s not necessary (but possible) to use it explicitly:

# default
default['apache']['dir'] = '/etc/apache2'

# override
override['apache']['dir'] = '/etc/apache2'

normal attribute

• a setting that persists in node object
• not reset before each chef-client run unlike other attributes

syntax in recipe or node file:

node.normal['apache']['dir'] = '/etc/apache2'
node.set['apache']['dir']    = '/etc/apache2' # set is an alias of normal
node['apache']['dir']        = '/etc/apache2' # same as above

syntax in cookbook attribute file:

# use of node object is implicit
normal['apache']['dir']      = '/etc/apache2'
set['apache']['dir']         = '/etc/apache2'

# use of node object is explicit
node.normal['apache']['dir'] = '/etc/apache2'
node.set['apache']['dir']    = '/etc/apache2'
node['apache']['dir']        = '/etc/apache2'

TODO: the same syntax node['cookbook']['attribute'] is used to read final node attribute value after all precedence rules have been applied.

ROLES

role

• server template (web, database, etc.)
• contains server-specific run-list and attributes
• create separate JSON file for each role in roles/

structure:

use role in node file:

{run_list: ['role[web]']}

ENVIRONMENTS

environment

• environment template (development, staging, production, etc.)
• contains environment-specific attributes
• doesn’t have run-list - attributes only
• create separate JSON file for each environment in environments/
• default environment is _default - all nodes are placed there unless another environment is specified
• each node can be in exactly one environment

structure:

set environment in node file:

{'run_list': ['recipe[apache2]'], 'chef_environment': 'development'}

or else environment can be set using -E knife argument:

$ knife solo cook -E development

DATA BAGS

data bag

• global variable stored as JSON data
• often used to store sensitive data (credentials, etc.)
• data bag consists of data bag items: DATA_BAG/DATA_BAG_ITEM.json

structure (according to official docs data bag item should contain only the contents of raw_data):

SITE COOKBOOK

cookbook

• fundamential unit of configuration and policy distribution

create custom cookbook:

$ cd site-cookbooks/
$ knife cookbook create COOKBOOK

structure:

ATTRIBUTE FILES

• when cookbook is run against a node attributes inside all attribute files are evaluated in the context of node object
• cookbook attributes are usually placed into attributes/default.rb - file name default.rb is just a convention here. if necessary attributes can be grouped into several attribute files with arbitrary names
• attribute files are loaded in alphabetical order except for default.rb - it’s loaded first (http://stackoverflow.com/questions/19628418)
• attributes that haven’t been loaded yet are not available in attribute files - you can load them manually with include_attribute directive

sample attribute file:

default['apache']['dir']          = '/etc/apache2'
default['apache']['listen_ports'] = [ '80','443' ]

default is attribute type here - it has nothing to do with attribute file name (see Chef Solo/ATTRIBUTES section for more information).

use of node object (node) is implicit here though it can be used explicitly (see Chef Solo/ATTRIBUTES section for more information).

RECIPES

recipe

• the most fundamental configuration element
• written in Ruby
• stored in cookbook
• consists of resources
• may be included in other recipes
• may depend on other recipes
• must be added to run-list before it can be used by chef-client

default recipe - default.rb (see Chef Solo/NODES for information about default recipe).

include other recipes

recipe can include other recipes from other cookbooks using include_recipe method:

include_recipe 'apache2::mod_ssl'

included recipe must be declared as dependency in metadata.rb:

depends 'apache2'

• included recipe is just inlined at the point where it’s included
• only the first inclusion within recipe is processed - subsequent ones are ignored

write to log from within a recipe

log levels:

• debug
• info
• warn
• error
• fatal

Chef::Log.info 'some useful information'

RESOURCES

resource

• describes desired state for configuration item
• resources are grouped into recipes

2 types of resources:

1. platform resource (built-in)

• available from chef-client directly
• doesn’t require a cookbook

1. custom resource (provided by cookbook)

• must be defined in resource file located in resources/
• can use platform resources in resource file
• used in a recipe in the same way as platform resource

usage and definition:

• resources are used in recipes (both platform and custom resources)
• resources are defined in resource files (custom resources only)

NOTE: this section describes how to use platform and custom resources. see custom resource files on how to define custom resources.

usage syntax:

TYPE 'NAME' do
  PROPERTY_NAME 'PROPERTY_VALUE'
  action :TYPE_OF_ACTION
end

• type:
  • platform resource types: package, template, service, etc.
  • on custom resource type - see custom resource files
• name (in general depends on specific resource type):
  • for platform resources dealing with files (directory, cookbook_file, etc.) this is usually the path of corresponding file on chef node.
  • on custom resource name - see custom resource files
• properties (1+) - most properties have default values
• actions (1+) - all actions have default values

only non-default properties and actions must be specified.

sample resource usage:

directory '/tmp/something' do
  owner 'root'
  group 'root'
  mode 00755
  action :create
end

popular platform resources

NOTE: for template resource transfer not ERB templates themselves but static files generated from those templates.

NOTE: for directory resource try not to use recursive option since group, owner and mode properties are not applied to parent directories - they are owned by root:root!

custom resource files

NOTE: LWRP/HWRP paradigm is replaced with custom resources - see Custom Resources in Chef Client 12.5 for details.

NOTE: definitions are also deprecated in favor of custom resources.

resource file

• declares properties of custom resource
• loads current properties if resource already exists
• defines all resource actions (the first one is default)
• resource name syntax: COOKBOOK_RESOURCE where COOKBOOK is a site cookbook name and RESOURCE is a resource file name (hyphens in cookbook name are replaced with underscores).

syntax:

resource_name :httpd

property :instance_name, String, name_property: true
property :name, RubyType, default: 'value'

default_action :action_2

load_current_value do
  # some Ruby
end

action :action_1 do
 # a mix of built-in Chef resources and Ruby
end

action :action_2 do
 # a mix of built-in Chef resources and Ruby
end

usage of all these methods and parameters is optional:

• resource_name overrides default resource name
• name_property: true makes this property to use resource name as its value
• load_current_value block loads current values for all properties
• default_action overrides default action

TEMPLATES

template

• ERB template

to use template:

• add template resource to recipe
• add corresponding ERB template file in templates/

ERB basics:

variables inside template file

1. defined in template resource’s variables parameter:

template '/etc/sudoers' do
  ...
  variables(
    {
      sudoers_groups: node[:authorization][:sudo][:groups],
      sudoers_users: node[:authorization][:sudo][:users]
    }
  )
end

these variables are accessible in template file as instance variables (@).

1. node object properties (using the same syntax as in recipe)

sample template file:

<% if @username.present? %>
  Site <%= node[:fqdn] %> welcomes new user <%= @username %>!
<% end %>

file specificity

specific template file might be targeted for specific platform.

template file is looked up in the following locations within templates/ (the first one that matches wins):

• host-FQDN/
• PLATFORM-VERSION/
• PLATFORM/
• default/

NOTE: FQDN must match literally!

partials

partial

• just a template file rendered from another (top-level) template file
• doesn’t have any special naming rules - it’s just an ordinary template file
• rendered with render command

render

• by default partial is looked up in current cookbook using normal rules (that is taking file specificity into account)
• instance variables (defined in template resource) are available in partial only when passed through variables option
• node object is available in partial automatically

render partial from top-level template file:

<%= render 'another_template.erb', variables: { user: @user } %>

METADATA.RB

metadata.rb

• lives at the top of each cookbook’s directory
• provides hints to chef server to deploy cookbook correctly