Chef - solo (chef-solo and knife-solo)
05 Jan 2016summary from book ‘Cooking infrastructure by Chef’ by Alexey Vasiliev and official chef documentation.
participants:
- chef node (deployment target - this is what configured, has chef-client or chef-solo installed)
- chef server (combined with chef node when using chef-solo)
- workstation (from which knife or knife-solo is run - can be remote host as well)
core principles:
- idempotence
-
thick clients, thin server
chef does as much work as possible on the node and as little as possible on the server
-
order matters
within a recipe resources are applied in the order they are listed
chef-client
- installed on each node managed by chef server
- used to bring the node into the expected state
CHEF SOLO
chef-solo
- open-source version of chef-client with limited functionality
- doesn’t require access to chef server
- requires that a cookbook be on the same physical disk as the node
- doesn’t support authentication and authorization
knife
- provides interface between local chef-repo and chef server
knife-solo
$ gem install knife-solo
- installs knife as dependency
- adds 5 subcommands to knife:
knife solo init
(create a new kitchen)knife solo prepare
(install chef-solo on target host)knife solo cook
(upload current kitchen to target host and run chef-solo there)knife solo bootstrap
(prepare + cook)knife solo clean
(remove uploaded kitchen from target host)
berkshelf
- manages cookbooks and their dependencies (like bundler for rubygems)
- librarian-chef - alternative
kitchen
- chef-repo
- located on workstation
- uploaded to chef server (or chef node when using chef-solo)
create a new kitchen in current directory:
$ knife solo init .
file structure:
directory or file | usage |
---|---|
.chef/ | stores .pem files and knife.rb |
cookbooks/ | vendor cookbooks installed with berkshelf |
data_bags/ | data bags |
environments/ | environments |
nodes/ | nodes |
roles/ | roles |
site-cookbooks/ | custom cookbooks |
Berksfile | like Gemfile for rubygems |
.chef/knife.rb
- chef-repo specific knife settings (primarily paths to cookbooks, node files, etc.)
- loaded every time knife is run
VENDOR COOKBOOKS
Berksfile:
cookbook 'apache2'
install cookbook globally into ~/.berkshelf/cookbooks/:
$ berks install
install cookbook both locally into cookbooks/ and globally into ~/.bershelf/cookbooks/:
$ berks vendor cookbooks
NOTE: command berks install --path cookbooks
does the same but is
deprecated.
it’s necessary to specify directory (cookbooks/) - otherwise berkshelf will install cookbook into berks-cookbooks/.
knife[:berkshelf_path]
option has no effect on where berkshelf installs
cookbooks
- most likely it’s used by knife to search for cookbooks while provisioning. but
since
berks vendor
command installs cookbook into ~/.bershelf/cookbooks/ as well I can’t figure out how you might end up having cookbook installed locally only (which would justify existence ofknife[:berkshelf_path]
option).
NODES
NOTE: nodes are configured in node files for chef-solo only - chef-client interacts with chef server to retrieve node configuration.
node
- contains node-specific attributes and run-list
- represents machine (physical, virtual, etc.)
- separate JSON file for each node in nodes/
- node file name (as a rule): DOMAIN.json
run_list
- main key in node file
- contains array of recipes and roles
- recipe name format (2 variants):
COOKBOOK::RECIPE
recipe[COOKBOOK::RECIPE]
- default.rb - default recipe of cookbook
- executed when only cookbook name is specified without specific recipe
- can be explicitly called with
COOKBOOK::default
sample node file:
{run_list: ['recipe[apache2]']}
ATTRIBUTES
attribute
- specific detail about node
- provided from the following locations:
- node files
- attribute files (in cookbook)
- recipes (in cookbook)
- roles
- environments
accessor methods are automatically defined for attributes in Ruby files (that is it’s possible to use dot syntax instead of hash keys):
default['apache']['dir'] = '/etc/apache2'
default.apache.dir = '/etc/apache2'
or else it’s possible to use symbols instead of strings as hash keys:
default['apache']['dir'] = '/etc/apache2'
default[:apache][:dir] = '/etc/apache2'
attribute precedence
in general attribute precedence is organized in such a way as:
- to enable cookbooks and roles to define attribute defaults
- for normal attributes to define the values that should be specific for a node
- and for override attributes to force a certain value, even when a node already has that value specified
attribute types
see also http://stackoverflow.com/a/20835500
- default
- force_default (cannot be set in node file)
- normal
- override
- force_override (cannot be set in node file)
- automatic (cannot be set manually, collected by Ohai)
attribute type is specified when using attributes by calling appropriate function of node object:
# default
node.default['apache']['dir'] = '/etc/apache2'
# override
node.override['apache']['dir'] = '/etc/apache2'
NOTE: cookbook attribute files are evaluated in the context of node object - that is why it’s not necessary (but possible) to use it explicitly:
# default
default['apache']['dir'] = '/etc/apache2'
# override
override['apache']['dir'] = '/etc/apache2'
normal attribute
- a setting that persists in node object
- not reset before each chef-client run unlike other attributes
syntax in recipe or node file:
node.normal['apache']['dir'] = '/etc/apache2'
node.set['apache']['dir'] = '/etc/apache2' # set is an alias of normal
node['apache']['dir'] = '/etc/apache2' # same as above
syntax in cookbook attribute file:
# use of node object is implicit
normal['apache']['dir'] = '/etc/apache2'
set['apache']['dir'] = '/etc/apache2'
# use of node object is explicit
node.normal['apache']['dir'] = '/etc/apache2'
node.set['apache']['dir'] = '/etc/apache2'
node['apache']['dir'] = '/etc/apache2'
TODO: the same syntax node['cookbook']['attribute']
is used to read final
node attribute value after all precedence rules have been applied.
ROLES
role
- server template (web, database, etc.)
- contains server-specific run-list and attributes
- create separate JSON file for each role in roles/
structure:
key | required? | description |
---|---|---|
name |
yes | unique name (usually the same as file name) |
description |
yes | |
chef_type |
yes | ‘role’ |
json_class |
yes | ‘Chef::Role’ |
default_attributes |
role-specific attributes, can be overidden in node files | |
override_attributes |
role-specific forced attributes, cannot be overidden in node files | |
run_list |
yes | role-specific run-list (may include other roles) |
env_run_lists |
environment-specific run-lists |
use role in node file:
{run_list: ['role[web]']}
ENVIRONMENTS
environment
- environment template (development, staging, production, etc.)
- contains environment-specific attributes
- doesn’t have run-list - attributes only
- create separate JSON file for each environment in environments/
- default environment is
_default
- all nodes are placed there unless another environment is specified - each node can be in exactly one environment
structure:
key | required? | description |
---|---|---|
name |
yes | unique name (usually the same as file name) |
description |
yes | |
chef_type |
yes | ‘environment’ |
json_class |
yes | ‘Chef::Environment’ |
default_attributes |
environment-specific attributes, can be overidden in node files | |
override_attributes |
environment-specific forced attributes, cannot be overidden in node files |
set environment in node file:
{'run_list': ['recipe[apache2]'], 'chef_environment': 'development'}
or else environment can be set using -E
knife argument:
$ knife solo cook -E development
DATA BAGS
data bag
- global variable stored as JSON data
- often used to store sensitive data (credentials, etc.)
- data bag consists of data bag items: DATA_BAG/DATA_BAG_ITEM.json
structure (according to official docs data bag item should contain only the
contents of raw_data
):
key | required? | description |
---|---|---|
name |
yes | unique name (usually the same as file name) |
chef_type |
yes | ‘data_bag_item’ |
json_class |
yes | ‘Chef::DataBagItem’ |
data_bag |
yes | data bag name |
raw_data |
yes | attributes of data bag item (id attribute is required) |
SITE COOKBOOK
cookbook
- fundamential unit of configuration and policy distribution
create custom cookbook:
$ cd site-cookbooks/
$ knife cookbook create COOKBOOK
structure:
directory or file | deprecated? |
---|---|
attributes/ | |
definitions/ | yes |
files/ | |
libraries/ | |
providers/ | yes |
recipes/ | |
resources/ | |
templates/ | |
metadata.rb |
ATTRIBUTE FILES
- when cookbook is run against a node attributes inside all attribute files are evaluated in the context of node object
- cookbook attributes are usually placed into attributes/default.rb - file name default.rb is just a convention here. if necessary attributes can be grouped into several attribute files with arbitrary names
- attribute files are loaded in alphabetical order except for default.rb - it’s loaded first (http://stackoverflow.com/questions/19628418)
- attributes that haven’t been loaded yet are not available in attribute files -
you can load them manually with
include_attribute
directive
sample attribute file:
default['apache']['dir'] = '/etc/apache2'
default['apache']['listen_ports'] = [ '80','443' ]
default
is attribute type here - it has nothing to do with attribute file name
(see Chef Solo/ATTRIBUTES section for more information).
use of node object (node
) is implicit here though it can be used explicitly
(see Chef Solo/ATTRIBUTES section for more information).
RECIPES
recipe
- the most fundamental configuration element
- written in Ruby
- stored in cookbook
- consists of resources
- may be included in other recipes
- may depend on other recipes
- must be added to run-list before it can be used by chef-client
default recipe - default.rb (see Chef Solo/NODES for information about default recipe).
include other recipes
recipe can include other recipes from other cookbooks using include_recipe
method:
include_recipe 'apache2::mod_ssl'
included recipe must be declared as dependency in metadata.rb:
depends 'apache2'
- included recipe is just inlined at the point where it’s included
- only the first inclusion within recipe is processed - subsequent ones are ignored
write to log from within a recipe
log levels:
- debug
- info
- warn
- error
- fatal
Chef::Log.info 'some useful information'
RESOURCES
resource
- describes desired state for configuration item
- resources are grouped into recipes
2 types of resources:
- platform resource (built-in)
- available from chef-client directly
- doesn’t require a cookbook
- custom resource (provided by cookbook)
- must be defined in resource file located in resources/
- can use platform resources in resource file
- used in a recipe in the same way as platform resource
usage and definition:
- resources are used in recipes (both platform and custom resources)
- resources are defined in resource files (custom resources only)
NOTE: this section describes how to use platform and custom resources. see custom resource files on how to define custom resources.
usage syntax:
TYPE 'NAME' do
PROPERTY_NAME 'PROPERTY_VALUE'
action :TYPE_OF_ACTION
end
- type:
- platform resource types:
package
,template
,service
, etc. - on custom resource type - see custom resource files
- platform resource types:
- name (in general depends on specific resource type):
- for platform resources dealing with files (
directory
,cookbook_file
, etc.) this is usually the path of corresponding file on chef node. - on custom resource name - see custom resource files
- for platform resources dealing with files (
- properties (1+) - most properties have default values
- actions (1+) - all actions have default values
only non-default properties and actions must be specified.
sample resource usage:
directory '/tmp/something' do
owner 'root'
group 'root'
mode 00755
action :create
end
popular platform resources
resource | description |
---|---|
script |
execute script using specified interpretor |
bash |
execute script using Bash interpretor |
ruby |
execute script using Ruby interpretor |
link |
create sym or hard links |
directory |
manage directories |
cookbook_file |
transfer files from subdirectory of files/ (PLATFORM or default for any platform) |
template |
transfer files from subdirectory of templates/ (PLATFORM or default for any platform) |
package |
install package (rpm, deb, etc.) |
gem_package |
install gem system-wide |
chef_gem |
install gem into the instance of Ruby dedicated to chef-client (can be required immediatelly after it’s installed) |
cron |
modify cron entries |
user |
manage users |
NOTE: for template
resource transfer not ERB templates themselves but
static files generated from those templates.
NOTE: for directory
resource try not to use recursive
option since
group
, owner
and mode
properties are not applied to parent directories -
they are owned by root:root
!
custom resource files
NOTE: LWRP/HWRP paradigm is replaced with custom resources - see Custom Resources in Chef Client 12.5 for details.
NOTE: definitions are also deprecated in favor of custom resources.
resource file
- declares properties of custom resource
- loads current properties if resource already exists
- defines all resource actions (the first one is default)
- resource name syntax:
COOKBOOK_RESOURCE
whereCOOKBOOK
is a site cookbook name andRESOURCE
is a resource file name (hyphens in cookbook name are replaced with underscores).
syntax:
resource_name :httpd
property :instance_name, String, name_property: true
property :name, RubyType, default: 'value'
default_action :action_2
load_current_value do
# some Ruby
end
action :action_1 do
# a mix of built-in Chef resources and Ruby
end
action :action_2 do
# a mix of built-in Chef resources and Ruby
end
usage of all these methods and parameters is optional:
resource_name
overrides default resource namename_property: true
makes this property to use resource name as its valueload_current_value
block loads current values for all propertiesdefault_action
overrides default action
TEMPLATES
template
- ERB template
to use template:
- add
template
resource to recipe - add corresponding ERB template file in templates/
ERB basics:
syntax | description |
---|---|
<%= %> |
expressions (=) |
<% %> |
statements (-) |
variables inside template file
- defined in
template
resource’svariables
parameter:
template '/etc/sudoers' do
...
variables(
{
sudoers_groups: node[:authorization][:sudo][:groups],
sudoers_users: node[:authorization][:sudo][:users]
}
)
end
these variables are accessible in template file as instance variables (@).
- node object properties (using the same syntax as in recipe)
sample template file:
<% if @username.present? %>
Site <%= node[:fqdn] %> welcomes new user <%= @username %>!
<% end %>
file specificity
specific template file might be targeted for specific platform.
template file is looked up in the following locations within templates/ (the first one that matches wins):
- host-FQDN/
- PLATFORM-VERSION/
- PLATFORM/
- default/
NOTE: FQDN must match literally!
partials
partial
- just a template file rendered from another (top-level) template file
- doesn’t have any special naming rules - it’s just an ordinary template file
- rendered with
render
command
render
- by default partial is looked up in current cookbook using normal rules (that is taking file specificity into account)
- instance variables (defined in template resource) are available in partial
only when passed through
variables
option - node object is available in partial automatically
render partial from top-level template file:
<%= render 'another_template.erb', variables: { user: @user } %>
METADATA.RB
metadata.rb
- lives at the top of each cookbook’s directory
- provides hints to chef server to deploy cookbook correctly
setting | description |
---|---|
name |
cookbook name (main setting) |
maintainer |
|
maintainer_email |
|
description |
|
long_description |
usually the contents of README.md |
version |
three-number version sequence |
chef_version |
range of chef-client versions supported by cookbook |
attribute |
attribute required to configure cookbook |
provides |
recipe or resource provided by cookbook |
recipe |
description for recipe (cosmetic value) |
depends |
cookbook dependency on another cookbook |
conflicts |
FIO. conflicting cookbook or cookbook version |
recommends |
FIO. recommended cookbook |
suggests |
FIO. suggested cookbook (weaker than recommends ) |
replaces |
FIO. cookbook to be replaced by this cookbook |
supports |
supported platform |