Puppet is a configuration management system. It is used to bring hosts to the desired state and maintain that state.

I have been working with Puppet for over five years. This text is essentially a translated and rearranged compilation of key points from the official documentation that will help newcomers quickly grasp the essence of Puppet.

Basic Information

The operation scheme of Puppet is client-server, although a serverless mode with limited functionality is also supported.

A pull model is used: by default, clients request configuration from the server every half hour and apply it. If you have worked with Ansible, it uses a different push model: the administrator initiates the process of applying the configuration, and clients will not apply anything by themselves.

Bidirectional TLS encryption is used for network interaction: both the server and the client have their own private keys and corresponding certificates. Usually, the server issues certificates for the clients, but it is generally possible to use an external CA as well.

Getting Started with Manifests

In Puppet terminology nodes connect to the Puppet server. Configuration for nodes is written in manifests in a special programming language — Puppet DSL. Puppet DSL is a declarative language. It describes the desired state of a node in the form of declarations of individual resources, for example: The file exists, and it has specific content.

The package is installed.

The service is running.
Resources can be interrelated:
There are dependencies, and they affect the order of resource application.

For example, 'first install the package, then modify the configuration file, after that start the service.'

There are notifications — if a resource changes, it sends notifications to the resources subscribed to it.
For instance, if the configuration file changes, the service can be automatically restarted.
Additionally, Puppet DSL has functions and variables, as well as conditional operators and selectors. Various templating mechanisms are also supported — EPP and ERB.
Puppet is written in Ruby, so many constructs and terms are taken from there. Ruby allows for extending Puppet — adding complex logic, new resource types, and functions.

In addition, Puppet DSL includes functions and variables as well as conditional operators and selectors. It also supports various templating mechanisms — EPP and ERB.

Puppet is written in Ruby, so many constructs and terms are derived from there. Ruby allows extending Puppet — writing complex logic, new resource types, and functions.

During Puppet's operation, manifests for each specific node on the server are compiled into the directory. The directory This is a list of resources and their relationships after calculating the values of functions, variables, and expanding conditional operators.

Syntax and Code Style

Here are sections of the official documentation that will help you understand the syntax if the provided examples are not sufficient:

Here is an example of what a manifest looks like:

# Комментарии пишутся, как и много где, после решётки.
#
# Описание конфигурации ноды начинается с ключевого слова node,
# за которым следует селектор ноды — хостнейм (с доменом или без)
# или регулярное выражение для хостнеймов, или ключевое слово default.
#
# После этого в фигурных скобках описывается собственно конфигурация ноды.
#
# Одна и та же нода может попасть под несколько селекторов. Про приоритет
# селекторов написано в статье про синтаксис описания нод.
node 'hostname', 'f.q.d.n', /regexp/ {
  # Конфигурация по сути является перечислением ресурсов и их параметров.
  #
  # У каждого ресурса есть тип и название.
  #
  # Внимание: не может быть двух ресурсов одного типа с одинаковыми названиями!
  #
  # Описание ресурса начинается с его типа. Тип пишется в нижнем регистре.
  # Про разные типы ресурсов написано ниже.
  #
  # После типа в фигурных скобках пишется название ресурса, потом двоеточие,
  # дальше идёт опциональное перечисление параметров ресурса и их значений.
  # Значения параметров указываются через т.н. hash rocket (=>).
  resource { 'title':
    param1 => value1,
    param2 => value2,
    param3 => value3,
  }
}

Whitespace and line breaks are not mandatory parts of the manifest, but there is a recommended style guide.Summary:

Double spaces, tabs are not used.
Curly braces are separated by a space, colons are not separated by spaces.
Commas after each parameter, including the last one. Each parameter should be on a separate line. An exception is made for cases without parameters and a single parameter: they can be written on one line and without a comma (i.e. resource { 'title': } and resource { 'title': param => value }).
Arrow signs for parameters should be aligned.
Resource relationship arrows are written before them.

File Location on the Puppet Server

For further explanations, I will introduce the concept of "root directory". The root directory is the directory where the Puppet configuration for a specific node resides.

The root directory varies depending on the Puppet version and the use of environments. Environments are independent sets of configurations stored in separate directories. They are usually used in combination with Git, in which case environments are created from Git branches. Accordingly, each node is in one environment or another. This is configured on the node itself or in the ENC, which I will discuss in the next article.

In the third version (the "old Puppet"), the base directory was /etc/puppet. The use of environments is optional - for example, we do not use them with the old Puppet. If environments are used, they are usually stored in /etc/puppet/environments, the root directory will be the environment directory. If environments are not used, the root directory will be the base one.
Starting from the fourth version (the "new Puppet"), the use of environments has become mandatory, and the base directory has been relocated to /etc/puppetlabs/code. Therefore, environments are stored in /etc/puppetlabs/code/environments, the root directory is the directory of the environment.

In the root directory, there should be a subdirectory manifests, which contains one or more manifests describing the nodes. Additionally, there should be a subdirectory modules, where the modules are located. I will explain what modules are a bit later. Additionally, in the old Puppet there might also be a subdirectory files, where various files are stored that we copy to the nodes. In the new Puppet, however, all files have been moved to modules.

Manifest files have the extension .pp.

A couple of real-world examples

Description of the node and the resource on it

On the node server1.testdomain there must be a file created /etc/issue with the content Debian GNU/Linux n l. The file must belong to the user and group root, the permissions must be 644.

Creating a manifest:

node 'server1.testdomain' {   # configuration block for node server1.testdomain
    file { 'etc/issue':   # describing the file /etc/issue
        ensure  => present,   # this file must exist
        content => 'Debian GNU/Linux n l',   # it must have this content
        owner   => root,   # owner user
        group   => root,   # owner group
        mode    => '0644',   # file permissions. They are specified as a string (in quotes) because otherwise a number starting with 0 will be interpreted as written in octal, which will lead to unexpected results
    }
}

Resource relationships on the node

On the node server2.testdomain must have nginx running with a pre-prepared configuration.

Decomposing the task:

It is necessary to have the package installed nginx.
It is necessary to copy the configuration files from the server.
It is necessary for the service to be running nginx.
In case of configuration updates, the service must be restarted.

Creating a manifest:

node 'server2.testdomain' {   # Configuration block related to the node server2.testdomain
    package { 'nginx':   # Describing the nginx package
        ensure => installed,   # It must be installed
    }
  # The direct arrow (->) indicates that the resource below should
  # be created after the resource described above.
  # Such dependencies are transitive.
    -> file { '/etc/nginx':   # Describing the file /etc/nginx
        ensure  => directory,   # This should be a directory
        source  => 'puppet:///modules/example/nginx-conf',   # Its content should be taken from the puppet server at the specified address
        recurse => true,   # Copy files recursively
        purge   => true,   # It should remove unnecessary files (those not in the source)
        force   => true,   # Remove unnecessary directories
    }
  # The wavy arrow (~>) indicates that the resource below should
  # subscribe to changes of the resource described above.
  # The wavy arrow includes the direct one (->).
    ~> service { 'nginx':   # Describing the nginx service
        ensure => running,   # It must be running
        enable => true,   # It should start automatically on system startup
    }
  # When a resource of the type service receives a notification,
  # the corresponding service gets restarted.
}

For this to work, you need a file structure like this on the puppet server:

/etc/puppetlabs/code/environments/production/ # (это для нового Паппета, для старого корневой директорией будет /etc/puppet)
├── manifests/
│   └── site.pp
└── modules/
    └── example/
        └── files/
            └── nginx-conf/
                ├── nginx.conf
                ├── mime.types
                └── conf.d/
                    └── some.conf

Resource Types

The complete list of supported resource types can be found in the documentation, here I will describe five basic types that are sufficient in my experience to solve most tasks.

file

Manages files, directories, symlinks, their content, and access rights.

Parameters:

resource name — file path (optional)
— Specifies the path in the container to which LXD will mount this device. — file path (if not specified in the name)
ensure — file type:
- absent — delete file
- present — must be a file of any type (if the file does not exist, a regular file will be created)
- file — regular file
- directory — directory
- link — symlink
content — file content (only suitable for regular files, cannot be used together with source or target)
source — reference to the path from which to copy the file content (cannot be used together with content or target). It can be specified as a URI with the schema puppet: (then files from the puppet server will be used), as well as with the schema http: (I hope it's clear what will happen in this case), and even with the schema file: or as an absolute path without a schema (in which case a file from the local FS on the node will be used)
target — where the symlink should point (cannot be used together with content or source)
owner — the user to whom the file should belong
group — the group to which the file should belong
mode — permissions for the file (as a string)
recurse — includes recursive processing of directories
purge — includes the removal of files that are not described in Puppet
force — includes the removal of directories that are not described in Puppet

package

Installs and removes packages. Can handle notifications—reinstalls the package if specified reinstall_on_refresh.

Parameters:

resource name — the name of the package (optional)
name — the name of the package (if not specified in the name)
provider — the package manager to use
ensure — the desired state of the package:
- present, installed — any version installed
- latest — the latest version installed
- absent — removed (apt-get remove)
- purged — removed along with configuration files (apt-get purge)
- held — the package version is held (apt-mark hold)
- any other string — the specified version installed
reinstall_on_refresh — if true, then upon notification, the package will be reinstalled. Useful for source-based distributions where rebuilding packages may be necessary when build parameters change. By default false.

cat << EOF | sudo tee -a /etc/systemd/system/lxd-hddpool.service [Unit] Description=Losetup LXD Storage Pool (hddpool) After=local-fs.target[Service] Type=oneshot ExecStart=/sbin/losetup /dev/loop1 /mnt/work/lxd/hddpool.img RemainAfterExit=true[Install] WantedBy=local-fs.target EOF

Manages services. Can handle notifications—restarts the service.

Parameters:

resource name — the service to manage (optional)
name — the service to manage (if not specified in the name)
ensure — the desired state of the service:
- running — running
- stopped — stopped
enable — manages the ability to start the service:
- true — auto-start is enabled (systemctl enable)
- mask — masked (systemctl mask)
- false — auto-start is disabled (systemctl disable)
restart — command to restart the service
status — command to check the service status
hasrestart — specify whether the service init script supports restart. If false and the option is given restart — the value of this parameter is used. If false and the parameter restart is not specified—the service stops and starts for restart (but in systemd the command used is systemctl restart).
hasstatus — specify whether the service init script supports the command status. If false, then the value of the parameter is used status. By default true.

exec

Runs external commands. If no parameters are specified creates, onlyif, unless or refreshonly, the command will run on every Puppet run. Can handle notifications—executes the command.

Parameters:

resource name — the command to execute (optional)
command — the command to be executed (if not specified in the name)
— Specifies the path in the container to which LXD will mount this device. — paths in which to search for the executable file
onlyif — if the command specified in this parameter exited with a zero return code, the main command will be executed
unless — if the command specified in this parameter exited with a non-zero return code, the main command will be executed
creates — if the file specified in this parameter does not exist, the main command will be executed
refreshonly — if true, the command will be executed only if this exec receives a notification from other resources
cwd — the directory from which to run the command
user — the user from whom to run the command
provider — using what to run the command:
- posix — simply creates a child process, must be specified — Specifies the path in the container to which LXD will mount this device.
- shell — the command is executed in the shell /bin/sh, can be omitted — Specifies the path in the container to which LXD will mount this device., globbing, pipes, and other shell features can be used. Usually determined automatically if there are various special characters (|, ;, &&, || and so on).

cron

Manages cron jobs.

Parameters:

resource name — just some identifier
ensure — the state of the cron job:
- present — create if it does not exist
- absent — delete if it exists
command — which command to run
environment — in what environment to run the command (list of environment variables and their values through =)
user — from which user to run the command
minute, hour, weekday, month, monthday — when to run cron. If any of these attributes are not specified, their value in the crontab will be *.

In Puppet 6.0 cron like it removed from the box in puppetserver, so there is no documentation on the general site. But it is in the box in puppet-agent, so there is no need to install it separately. Documentation for it can be viewed in the documentation for the fifth version of Puppet, or on GitHub.

About resources in general

Requirements for uniqueness of resources

The most common mistake we encounter — Duplicate declaration. This error occurs when two or more resources of the same type with the same name enter the directory.

Therefore, I will repeat: in manifests for one node, there should not be resources of the same type with the same name (title)!

Sometimes there is a need to install packages with the same name, but different package managers. In this case, you need to use the parameter name, to avoid the error:

package { 'ruby-mysql':
  ensure   => installed,
  name     => 'mysql',
  provider => 'gem',
}
package { 'python-mysql':
  ensure   => installed,
  name     => 'mysql',
  provider => 'pip',
}

Other types of resources have similar parameters that help avoid duplication, - name at cat << EOF | sudo tee -a /etc/systemd/system/lxd-hddpool.service [Unit] Description=Losetup LXD Storage Pool (hddpool) After=local-fs.target[Service] Type=oneshot ExecStart=/sbin/losetup /dev/loop1 /mnt/work/lxd/hddpool.img RemainAfterExit=true[Install] WantedBy=local-fs.target EOF, command at exec, and so on.

Metaparameters

Some special parameters exist for each resource type, regardless of its entity.

Complete list of metaparameters in the Puppet documentation.

Summary list:

require — this parameter specifies the resources that this resource depends on.
before — this parameter specifies the resources that depend on this resource.
subscribe — this parameter specifies the resources from which this resource receives notifications.
notify — this parameter specifies which resources receive notifications from this resource.

All the listed metaparameters accept either a single reference to a resource or an array of references in square brackets.

References to resources

A reference to a resource is simply a mention of the resource. They are mainly used to indicate dependencies. A reference to a non-existent resource will cause a compilation error.

The syntax for a reference is as follows: resource type with an uppercase letter (if the type name contains double colons, each part of the name between the colons is capitalized), then in square brackets the name of the resource (the case of the name is unchanged!). No spaces should be present, the square brackets are written immediately after the type name.

Example:

file { '/file1': ensure => present }
file { '/file2':
  ensure => directory,
  before => File['/file1'],
}
file { '/file3': ensure => absent }
File['/file1'] -> File['/file3']

Dependencies and notifications

Documentation is here.

As mentioned earlier, simple dependencies between resources are transitive. Be careful when setting dependencies — it's possible to create circular dependencies, which will cause a compilation error.

Unlike dependencies, notifications are not transitive. The following rules apply to notifications:

If a resource receives a notification, it is updated. The actions taken upon updating depend on the resource type — exec executes a command, cat << EOF | sudo tee -a /etc/systemd/system/lxd-hddpool.service [Unit] Description=Losetup LXD Storage Pool (hddpool) After=local-fs.target[Service] Type=oneshot ExecStart=/sbin/losetup /dev/loop1 /mnt/work/lxd/hddpool.img RemainAfterExit=true[Install] WantedBy=local-fs.target EOF restarts a service, package reinstalls a package. If no action is defined for the resource upon updating, nothing happens.
In a single run, a Puppet resource is updated no more than once. This is possible because notifications include dependencies, and the dependency graph does not contain cycles.
If Puppet changes the state of a resource, the resource sends notifications to all other resources that are subscribed to it.
If a resource is updated, it sends notifications to all resources that are subscribed to it.

Handling unspecified parameters

Generally, if a resource parameter has no default value and this parameter is not specified in the manifest, Puppet will not change that property of the corresponding resource on the node. For example, if a resource of type file does not specify a parameter owner, then Puppet will not change the owner of the corresponding file.

Introduction to classes, variables, and defines

Suppose we have several nodes that share the same configuration parts but also have differences — otherwise, we could describe everything in a single block. node {}. Of course, one can simply copy identical parts of the configuration, but in general, this is a bad solution — the configuration expands, and when changing the shared part, one has to modify the same thing in many places. It is easy to make a mistake, and the DRY principle (don’t repeat yourself) exists for a reason.

To solve such a problem, there is a construct called class.

Classes

Class — a named block of Puppet code. Classes are needed for code reuse.

First, a class needs to be defined. The definition itself does not add any resources anywhere. A class is defined in manifests:

# Описание класса начинается с ключевого слова class и его названия.
# Дальше идёт тело класса в фигурных скобках.
class example_class {
    ...
}

After that, the class can be used:

# первый вариант использования — в стиле ресурса с типом class
class { 'example_class': }
# второй вариант использования — с помощью функции include
include example_class
# про отличие этих двух вариантов будет рассказано дальше

As an example from the previous task, let’s extract the installation and configuration of nginx into a class:

class nginx_example {
    package { 'nginx':
        ensure => installed,
    }
    -> file { '/etc/nginx':
        ensure => directory,
        source => 'puppet:///modules/example/nginx-conf',
        recurse => true,
        purge  => true,
        force  => true,
    }
    ~> service { 'nginx':
        ensure => running,
        enable => true,
    }
}

node 'server2.testdomain' {
    include nginx_example
}

Variables

The class from the previous example is not very flexible, as it always brings the same nginx configuration. Let’s make the path to the configuration variable so that this class can be used to install nginx with any configuration.

This can be done using variables.

Attention: variables in Puppet are immutable!

Moreover, a variable can only be accessed after it has been declared; otherwise, its value will be undef.

Example of working with variables:

# создание переменных
$variable = 'value'
$var2 = 1
$var3 = true
$var4 = undef
# использование переменных
$var5 = $var6
file { '/tmp/text': content => $variable }
# интерполяция переменных — раскрытие значения переменных в строках. Работает только в двойных кавычках!
$var6 = "Variable with name variable has value ${variable}"

In Puppet, there are namespaces, and variables, respectively, have scope: a variable with the same name can be defined in different namespaces. When resolving the value of a variable, the variable is searched for in the current namespace first, then in the encompassing one, and so on.

Examples of namespaces:

global — this includes variables outside the class or node declaration;
node namespace in the node declaration;
class namespace in the class declaration.

To avoid ambiguity when referring to a variable, you can specify the namespace in the variable name:

# переменная без пространства имён
$var
# переменная в глобальном пространстве имён
$::var
# переменная в пространстве имён класса
$classname::var
$::classname::var

Let's agree that the path to the nginx configuration is stored in the variable $nginx_conf_source. Then the class will look as follows:

class nginx_example {
    package { 'nginx':
        ensure => installed,
    }
    -> file { '/etc/nginx':
        ensure => directory,
        source => $nginx_conf_source,   # here we use the variable instead of a fixed string
        recurse => true,
        purge  => true,
        force  => true,
    }
    ~> service { 'nginx':
        ensure => running,
        enable => true,
    }
}

node 'server2.testdomain' {
    $nginx_conf_source = 'puppet:///modules/example/nginx-conf'
    include nginx_example
}

However, the example given is poor because there is some 'secret knowledge' that a variable with a certain name is used somewhere inside the class. It would be much better to make this knowledge common — classes can have parameters.

Class parameters — these are variables in the class namespace, defined in the class header and can be used like regular variables inside the class body. Parameter values are specified when using the class in a manifest.

A parameter can have a default value. If a parameter does not have a default value and one is not provided during use, it will trigger a compilation error.

Let's parameterize the class from the above example and add two parameters: the first, mandatory — the path to the configuration, and the second, optional — the name of the package with nginx (in Debian, for example, there are packages nginx, nginx-light, nginx-full).

# переменные описываются сразу после имени класса в круглых скобках
class nginx_example (
  $conf_source,
  $package_name = 'nginx-light', # параметр со значением по умолчанию
) {
  package { $package_name:
    ensure => installed,
  }
  -> file { '/etc/nginx':
    ensure  => directory,
    source  => $conf_source,
    recurse => true,
    purge   => true,
    force   => true,
  }
  ~> service { 'nginx':
    ensure => running,
    enable => true,
  }
}

node 'server2.testdomain' {
  # если мы хотим задать параметры класса, функция include не подойдёт* — нужно использовать resource-style declaration
  # *на самом деле подойдёт, но про это расскажу в следующей серии. Ключевое слово "Hiera".
  class { 'nginx_example':
    conf_source => 'puppet:///modules/example/nginx-conf',   # задаём параметры класса точно так же, как параметры для других ресурсов
  }
}

In Puppet, variables are typed. There are many data typesData types are typically used to validate the values of parameters passed to classes and defines. If the passed parameter does not match the specified type, a compilation error will occur.

The type is written directly before the parameter name:

class example (
  String $param1,
  Integer $param2,
  Array $param3,
  Hash $param4,
  Hash[String, String] $param5,
) {
  ...
}

Classes: include classname vs class{'classname':}

Each class is a resource type class. Just like with any other types of resources, there cannot be two instances of the same class on one node.

If you try to add a class to the same node twice using class { 'classname':} (regardless of whether with different or the same parameters), there will be a compilation error. However, when using a class in resource style, you can explicitly set all its parameters right in the manifest.

However, if you use include, then the class can be added an unlimited number of times. The thing is that include is an idempotent function that checks if the class is in the catalog. If the class is not in the catalog, it adds it; if it already exists, it does nothing. But in the case of using include you cannot set class parameters at the time of class declaration—all mandatory parameters must be specified in an external data source—Hiera or ENC. We will discuss these in the next article.

Defines

As mentioned in the previous section, the same class cannot be present on a node more than once. However, in some cases, there is a need to apply the same block of code with different parameters on one node. In other words, there is a need for a custom resource type.

For example, to install a PHP module, we do the following in Avito:

We install the package for this module.
We create a configuration file for this module.
We create a symlink to the config for php-fpm.
We create a symlink to the config for php cli.

In such cases, a construction called define (define, defined type, defined resource type) is used. A define is similar to a class, but there are differences: first, each define is a type of resource, not a resource itself; second, each define has an implicit parameter $title, which receives the resource name upon declaration. Just like with classes, a define needs to be described first before it can be used.

Simplified example with a PHP module:

define php74::module (
  $php_module_name = $title,
  $php_package_name = "php7.4-${title}",
  $version = 'installed',
  $priority = '20',
  $data = "extension=${title}.son",
  $php_module_path = 'etc/php/7.4/mods-available',
) {
  package { $php_package_name:
    ensure          => $version,
    install_options => ['-o', 'DPkg::NoTriggers=true'],  # Debian PHP package triggers automatically create symlinks and restart the php-fpm service - we don’t need this because we manage both symlinks and the service using Puppet
  }
  -> file { "${php_module_path}/${php_module_name}.ini":
    ensure  => $ensure,
    content => $data,
  }
  file { "etc/php/7.4/cli/conf.d/${priority}-${php_module_name}.ini":
    ensure  => link,
    target  => "${php_module_path}/${php_module_name}.ini",
  }
  file { "etc/php/7.4/fpm/conf.d/${priority}-${php_module_name}.ini":
    ensure  => link,
    target  => "${php_module_path}/${php_module_name}.ini",
  }
}

node server3.testdomain {
  php74::module { 'sqlite3': }
  php74::module { 'amqp': php_package_name => 'php-amqp' }
  php74::module { 'msgpack': priority => '10' }
}

In the define, it's easiest to catch a Duplicate declaration error. This happens if there is a resource with a constant name in the define, and on some node, there are two or more instances of this define.

It's easy to protect against this: all resources inside the define should have a name that depends on $title. As an alternative - idempotent resource addition, in the simplest case, it's sufficient to move common resources for all instances of the define to a separate class and include this class in the define - the function include is idempotent.

There are other ways to achieve idempotency when adding resources, namely the use of functions defined and ensure_resources, but I'll cover that in the next series.

Dependencies and notifications for classes and defines

Classes and defines add the following rules for handling dependencies and notifications:

a dependency on a class/define adds dependencies for all resources of the class/define;
a class/define dependency adds dependencies to all resources of the class/define;
a class/define notification notifies all resources of the class/define;
a subscription to a class/define subscribes to all resources of the class/define.

Conditional operators and selectors

Documentation is here.

if

It's straightforward here:

if CONDITION1 {
  ...
} elsif CONDITION2 {
  ...
} else {
  ...
}

unless

unless is the opposite of if: the code block will be executed if the expression is false.

unless CONDITION {
  ...
}

case

Nothing complex here, either. In terms of values, you can use regular values (strings, numbers, etc.), regular expressions, as well as data types.

case EXPRESSION {
  VALUE1: { ... }
  VALUE2, VALUE3: { ... }
  default: { ... }
}

Selectors

A selector is a language construct similar to case, but instead of executing a block of code, it returns a value.

$var = $othervar ? { 'val1' => 1, 'val2' => 2, default => 3 }

Modules

When the configuration is small, it can easily be kept in one manifest. However, as we describe more configuration, the number of classes and nodes in the manifest grows, making it unwieldy to work with.

Moreover, there is the issue of code reuse — when all the code is in one manifest, it is difficult to share that code with others. To address these two problems, Puppet has an entity called modules.

Modules — these are sets of classes, defines, and other Puppet entities placed in a separate directory. In other words, a module is an independent piece of Puppet logic. For example, there may be a module for working with nginx, which will contain only what is necessary to work with nginx, and there may be a module for working with PHP, and so on.

Modules are versioned and module dependencies on each other are also supported. There is an open repository for modules — Puppet Forge.

On the Puppet server, modules are located in the modules subdirectory of the root directory. Inside each module, there is a standard directory scheme — manifests, files, templates, lib, and so on.

File structure in a module

At the root of the module, there may be the following directories with meaningful names:

manifests — contains the manifests
files — contains the files
templates — contains the templates
lib — contains Ruby code

This is not a complete list of directories and files, but it is sufficient for this article.

Resource names and file names in a module

Documentation is here.

Resources (classes, defines) in a module cannot be named arbitrarily. Additionally, there is a direct correspondence between the resource name and the filename in which Puppet will look for the description of that resource. If naming rules are violated, Puppet simply won't find the resource descriptions, resulting in a compilation error.

The rules are simple:

All resources in a module must be in the module's namespace. If the module is named foo, then all resources in it must be named foo::, or simply foo.
A resource with the name of the module must be in the file init.pp.
For other resources, the file naming scheme is as follows:
- the prefix with the module name is dropped
- All double colons, if present, are replaced with slashes.
- An extension is added. .pp

I'll demonstrate with an example. Let's say I'm writing a module. nginxIt includes the following resources:

class nginx described in the manifest. init.pp;
class nginx::service described in the manifest. service.pp;
define nginx::server described in the manifest. server.pp;
define nginx::server::location described in the manifest. server/location.pp.

Templates

Surely you know what templates are; I won’t describe them in detail here. But just in case, I'll leave a link to Wikipedia..

How to use templates: the value of the template can be revealed using the function template, which takes the path to the template. For resources like file it is used together with the parameter content. For example, like this:

file { '/tmp/example': content => template('modulename/templatename.erb')

A path like / implies a file at /modules//templates/.

Additionally, there is a function inline_template — it takes the template text as input, not the file name.

Inside templates, you can use all Puppet variables in the current scope.

Puppet supports templates in ERB and EPP format:

A brief overview of ERB.

Control structures:

<%= ВЫРАЖЕНИЕ %> — insert the value of the expression.
<% ВЫРАЖЕНИЕ %> — evaluate the expression (without inserting it). This is where conditional operators (if) and loops (each) usually go.
<%# КОММЕНТАРИЙ %>

Expressions in ERB are written in Ruby (in fact, ERB stands for Embedded Ruby).

To access variables from the manifest, you need to prepend @ to the variable name. To remove the newline that appears after the control structure, you need to use the closing tag -%>.

Example of template usage.

Let's say I'm writing a module to manage ZooKeeper. The class responsible for creating the config looks something like this:

class zookeeper::configure (
  Array[String] $nodes,
  Integer $port_client,
  Integer $port_quorum,
  Integer $port_leader,
  Hash[String, Any] $properties,
  String $datadir,
) {
  file { '/etc/zookeeper/conf/zoo.cfg':
    ensure  => present,
    content => template('zookeeper/zoo.cfg.erb'),
  }
}

And the corresponding template zoo.cfg.erb — is like this:

0 -%>

server. = :::



dataDir = 


 =

Facts and built-in variables.

Often, a specific part of the configuration depends on what is currently happening on the node. For example, depending on which version of Debian is installed, one needs to install a specific version of a package. You can monitor all of this manually by rewriting the manifests in case of node changes. However, this is an unserious approach; automation is much better.

To obtain information about nodes in Puppet, there is a mechanism called facts. Facts are information about the node available in manifests as regular variables in the global namespace. For example, the hostname, operating system version, CPU architecture, list of users, list of network interfaces and their addresses, and much, much more. Facts are available in manifests and templates as regular variables.

Example of working with facts:

notify { "Running OS ${facts['os']['name']} version ${facts['os']['release']['full']}": }
# The notify resource simply outputs a message to the log

Formally speaking, a fact has a name (string) and a value (various types are available: strings, arrays, dictionaries). There is a set of built-in facts. You can also write your own. Fact collectors are described as functions in Ruby, or as executable files. Additionally, facts can be presented as text files with data on the nodes.

During operation, the Puppet agent first copies all available fact collectors from the Puppet server to the node, then runs them and sends the collected facts back to the server; only after this does the server begin compiling the catalog.

Facts as executable files

These facts are placed in modules in the facts.ddirectory. Naturally, the files must be executable. When executed, they should output information to standard output either in YAML format or in a "key=value" format.

Do not forget that facts are distributed to all nodes managed by the Puppet server to which your module is deployed. Therefore, make sure in your script that all programs and files necessary for the operation of your fact are present in the system.

#!/bin/sh
echo "testfact=success"

#!/bin/sh
echo '{"testyamlfact":"success"}'

Facts in Ruby

These facts are placed in modules in the directory lib/facter.

# всё начинается с вызова функции Facter.add с именем факта и блоком кода
Facter.add('ladvd') do
# в блоках confine описываются условия применимости факта — код внутри блока должен вернуть true, иначе значение факта не вычисляется и не возвращается
  confine do
    Facter::Core::Execution.which('ladvdc') # проверим, что в PATH есть такой исполняемый файл
  end
  confine do
    File.socket?('/var/run/ladvd.sock') # проверим, что есть такой UNIX-domain socket
  end
# в блоке setcode происходит собственно вычисление значения факта
  setcode do
    hash = {}
    if (out = Facter::Core::Execution.execute('ladvdc -b'))
      out.split.each do |l|
        line = l.split('=')
        next if line.length != 2
        name, value = line
        hash[name.strip.downcase.tr(' ', '_')] = value.strip.chomp(''').reverse.chomp(''').reverse
      end
    end
    hash  # значение последнего выражения в блоке setcode является значением факта
  end
end

Text facts

These facts are placed on the nodes in the directory /etc/facter/facts.d in the old Puppet or /etc/puppetlabs/facts.d in the new Puppet.

examplefact=examplevalue

---
examplefact2: examplevalue2
anotherfact: anothervalue

Accessing facts

There are two ways to access facts:

through the dictionary $facts: $facts['fqdn'];
using the fact name as the variable name: $fqdn.

It is best to use the dictionary $facts, and even better to specify the global namespace ($::facts).

Here is the required section of the documentation.

Built-in variables

In addition to facts, there are also some variables, available in the global namespace.

trusted facts — variables taken from the client's certificate (since the certificate is usually issued on the puppet server, the agent cannot just change its certificate, so the variables are 'trusted'): the certificate name, host name, and domain, extensions from the certificate.
server facts — variables related to server information — version, name, server IP address, environment.
agent facts — variables added directly by the puppet-agent, not by facter — certificate name, agent version, puppet version.
master variables — variables of the puppet master (sic!). They are roughly the same as in server facts, plus configuration parameter values are available.
compiler variables — compiler variables that differ in each scope: the name of the current module and the name of the module in which the current object's reference occurred. They can be used, for example, to check that your private classes are not used directly from other modules.

Appendix 1: How to run and debug all of this?

The article contains many examples of puppet code, but it does not tell how to run this code. Well, I'm correcting that.

For Puppet to work, an agent is sufficient, but in most cases, a server will also be needed.

Agent

At least from version five, the puppet-agent packages from the official Puppetlabs repository contain all dependencies (ruby and the corresponding gems), so there are no installation issues (I’m referring to Debian-based distributions — we don’t use RPM-based distributions).

In the simplest case, to apply the puppet configuration, it is enough to run the agent in serverless mode: provided that the puppet code is copied to the node, you run puppet apply:

atikhonov@atikhonov ~/puppet-test $ cat helloworld.pp 
node default {
    notify { 'Hello world!': }
}
atikhonov@atikhonov ~/puppet-test $ puppet apply helloworld.pp 
Notice: Compiled catalog for atikhonov.localdomain in environment production in 0.01 seconds
Notice: Hello world!
Notice: /Stage[main]/Main/Node[default]/Notify[Hello world!]/message: defined 'message' as 'Hello world!'
Notice: Applied catalog in 0.01 seconds

It is better, of course, to elevate the server and run agents on the nodes in daemon mode — then every half an hour they will apply the configuration downloaded from the server.

You can simulate a push model of operation — access the node of interest and run sudo puppet agent -t. The option -t (--test) actually includes several options that can be enabled individually. Among these options are the following:

not operate in daemon mode (by default, the agent runs in daemon mode);
terminate after applying the catalog (by default, the agent continues to operate and applies the configuration every half hour);
write detailed logs of operations;
show changes in files.

The agent has a no-change mode — it can be used when you are not sure that you wrote the correct configuration and want to check what exactly the agent will change during its operation. This mode is enabled by the parameter --noop in the command line: sudo puppet agent -t --noop.

Additionally, you can enable a debug log — in it, Puppet writes about all actions it performs: about the resource that it is currently processing, about the parameters of this resource, and about which programs it runs. Naturally, this is the parameter --debug.

Server

I will not discuss full Puppet server setup and code deployment in this article, I will only say that a fully operational version of the server is installed out of the box, requiring no additional configuration to operate under a small number of nodes (say, up to one hundred). A larger number of nodes will require tuning — by default, the puppetserver runs no more than four workers, for greater performance you need to increase their number and remember to increase the memory limits, otherwise the server will spend most of its time garbage collecting.

For code deployment — if you need something quick and simple, check (on r10k)[https://github.com/puppetlabs/r10k], for small installations it should be sufficient.

Appendix 2: recommendations for writing code

Move all logic into classes and defines.
Keep classes and defines in modules, not in manifests with node descriptions.
Use the facts.
Avoid using if statements based on hostnames.
Don't hesitate to add parameters for classes and defines — it's better than having implicit logic hidden in the body of a class/define.

I will explain why I recommend this in the next article.

Conclusion

Let's conclude the introduction here. In the next article, I will talk about Hiera, ENC, and PuppetDB.

Only registered users can participate in the survey. Please log in, please.

In fact, there is much more material — I can write articles on the following topics, vote on what you would be interested in reading about:

59,1%Advanced Puppet constructs — some next-level stuff: loops, mapping, and other lambda expressions, resource collectors, exported resources, and inter-host communication via Puppet, tags, providers, abstract data types.
31,8%"I am an admin from my mom" or how we at Avito integrated several Puppet servers of different versions, along with general admin topics for Puppet servers.
81,8%How we write Puppet code: tooling, documentation, testing, CI/CD.

22 users voted. 9 users abstained.

Source: habr.com

Introduction to Puppet

Basic Information

Getting Started with Manifests

Syntax and Code Style

File Location on the Puppet Server

A couple of real-world examples

Description of the node and the resource on it

Resource relationships on the node

Resource Types

file

package

cat << EOF | sudo tee -a /etc/systemd/system/lxd-hddpool.service [Unit] Description=Losetup LXD Storage Pool (hddpool) After=local-fs.target[Service] Type=oneshot ExecStart=/sbin/losetup /dev/loop1 /mnt/work/lxd/hddpool.img RemainAfterExit=true[Install] WantedBy=local-fs.target EOF

exec

cron

About resources in general

Requirements for uniqueness of resources

Metaparameters

References to resources

Dependencies and notifications

Handling unspecified parameters

Introduction to classes, variables, and defines

Classes

Variables

Classes: include classname vs class{'classname':}

Defines

Dependencies and notifications for classes and defines

Conditional operators and selectors

if

unless

case

Selectors

Modules

File structure in a module

Resource names and file names in a module

Templates

A brief overview of ERB.

Example of template usage.

Facts and built-in variables.

Facts as executable files

Facts in Ruby

Text facts

Accessing facts

Built-in variables

Appendix 1: How to run and debug all of this?

Agent

Server

Appendix 2: recommendations for writing code

Conclusion

In fact, there is much more material — I can write articles on the following topics, vote on what you would be interested in reading about: