Hello everyone!

I work as a DevOps engineer at a hotel booking service. Ostrovok.ru. In this article, I want to talk about our experience with testing Ansible roles.

At Ostrovok.ru, we use Ansible as our configuration management tool. Recently, we found the need to test roles, but, as it turned out, there aren't many tools available for this — the most popular is probably the Molecule framework, so we decided to use it. However, its documentation omits many pitfalls. We couldn't find a sufficiently detailed guide in Russian, so we decided to write this article.

Molecule

Molecule — a framework to assist in testing Ansible roles.

Simplified description: Molecule creates an instance on the platform you specify (cloud, virtual machine, container; see section Driver), runs your role on it, then executes tests, and finally deletes the instance. If a failure occurs at any step, Molecule will notify you.

Now in more detail.

A Bit of Theory

Let's consider the two key entities of Molecule: Scenario and Driver.

Scenario

A scenario contains a description of what, where, how, and in what order will be executed. One role can have multiple scenarios, and each is a directory at the path /molecule/, containing descriptions of the required actions for the test. There must be a scenario default, which will be automatically created if you initialize the role using Molecule. The names of the subsequent scenarios are at your discretion.

The sequence of testing actions in a scenario is called matrix, and by default, it is as follows:

(Steps marked ?, by default, are skipped unless specified by the user)

• lint — running linters. By default, it uses yamllint and flake8,
• destroy — removing instances from the previous run of Molecule (if any remain),
• dependency? — установка ansible-зависимости тестируемой роли,
• syntax — checking the syntax of the role using ansible-playbook --syntax-check,
• create — creating an instance,
• prepare? — подготовка инстанса; например, проверка / установка python2
• converge — running the playbook under test,
• idempotence — rerunning the playbook to test for idempotence,
• side_effect? — действия, не относящиеся непосредственно к роли, но нужные для тестов,
• verify — running tests on the resulting configuration using testinfra(by default) /goss/inspec,
• Cleanup? — (в новых версиях) — грубо говоря, «очистка» внешней инфраструктуры, задетой Молекулой,
• destroy — removing the instance.

This sequence covers most cases, but it can be modified if necessary.

Each of the steps mentioned above can be run separately using molecule. However, it's important to understand that each CLI command may have its own sequence of actions, which can be learned by executing molecule matrix. For example, when running the command converge (running the test playbook), the following actions will be performed:

$ molecule matrix converge
...
└── default         # scenario name
    ├── dependency  # install dependencies
    ├── create      # create instance
    ├── prepare     # prepare instance
    └── converge    # run playbook

The sequence of these actions can be edited. If something from the list has already been completed, it will be skipped. Current state and instance configurations are stored by Molecule in the directory $TMPDIR/molecule//.

You can add steps with ? , describing the desired actions in the ansible-playbook format, and name the files according to the steps: prepare.yml/side_effect.yml. Molecule will expect these files to be in the scenario folder.

Driver

A driver is an entity where instances for tests are created.
The list of standard drivers for which Molecule has templates ready is as follows: Azure, Docker, EC2, GCE, LXC, LXD, OpenStack, Vagrant, Delegated.

In most cases, the templates are files create.yml and destroy.yml in the scenario folder that describe the creation and deletion of instances respectively.
Exceptions are Docker and Vagrant, as interactions with their modules can occur without the aforementioned files.

It's worth highlighting the Delegated driver, as when using it, the files for creating and deleting instances describe only the configuration work; the rest must be defined by the engineer.

The default driver is Docker.

Now let's move to practice and consider further details there.

Getting Started

As a 'hello world', we will test a simple role for installing nginx. We'll choose Docker as the driver — I believe it's installed on most of your machines (and remember, Docker is the default driver).

Let's prepare virtualenv and install molecule:

> pip install virtualenv
> virtualenv -p `which python2` venv
> source venv/bin/activate
> pip install molecule docker  # molecule will install ansible as a dependency; docker for the driver

As the next step, we initialize a new role.
The initialization of a new role, like a new scenario, is done using the command molecule init:

> molecule init role -r nginx
--> Initializing new role nginx...
Initialized role in /nginx successfully.
> cd nginx
> tree -L 1
.
├── README.md
├── defaults
├── handlers
├── meta
├── molecule
├── tasks
└── vars

6 directories, 1 file

A typical Ansible role has been created. All interactions with the Molecule CLI will take place from the root of the role.

Let's take a look at the contents of the role directory:

> tree molecule/default/
molecule/default/
├── Dockerfile.j2  # Jinja template for Dockerfile
├── INSTALL.rst.   # Some information about installing the script dependencies
├── molecule.yml   # Configuration file
├── playbook.yml   # Playbook to run the role
└── tests          # Directory with verification stage tests
    └── test_default.py

1 directory, 6 files

Let's analyze the config molecule/default/molecule.yml (we will only change the docker image):

---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: instance
    image: centos:7
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
verifier:
  name: testinfra
  lint:
    name: flake8

dependency

This section describes the source of dependencies.

Possible options include: galaxy, gilt, shell.

Shell is simply a command shell used when galaxy and gilt do not meet your needs.

I won’t dwell on this for long, it’s sufficiently described in the documentation.

driver

The name of the driver. In our case, it is docker.

lint

Yamllint is used as the linter.

Useful options in this part of the config include the ability to specify a configuration file for yamllint, pass environment variables, or disable the linter:

lint:
  name: yamllint
  options:
    config-file: foo/bar
  env:
    FOO: bar
  enabled: False

platforms

Describes the configuration of instances.
In the case of docker as the driver role, Molecule iterates over this section, and each list item is available in Dockerfile.j2 as the variable item.

In the case of a driver where create.yml and destroy.yml, the section is available in them as molecule_yml.platforms, and the iterations over it are described in those files.

Since Molecule provides management of instances via Ansible modules, the list of possible settings should also be sought there. For docker, for example, the module used is docker_container_module. Which modules are used in other drivers can be found in the documentation.

Examples of using various drivers can also be found in the tests of Molecule itself.

We'll replace it here centos:7 to ubuntu.

provisioner,

"Provider" is the entity that manages instances. In the case of Molecule, it is Ansible; support for others is not planned, so this section can be tentatively termed as advanced Ansible configuration.
Here you can specify a lot of things, I will highlight the main points, in my opinion:

• playbooks: you can specify which playbooks should be used at certain stages.

provisioner:
  name: ansible
  playbooks:
    create: create.yml
    destroy: ../default/destroy.yml
    converge: playbook.yml
    side_effect: side_effect.yml
    cleanup: cleanup.yml

• config_options: ansible config

provisioner:
  name: ansible
  config_options:
    defaults:
      fact_caching: jsonfile
    ssh_connection:
      scp_if_ssh: True

• connection_options: parameters of the connection

provisioner:
  name: ansible  
  connection_options:
    ansible_ssh_common_args: "-o 'UserKnownHostsFile=/dev/null' -o 'ForwardAgent=yes'"

• options: Ansible parameters and environment variables

provisioner:
  name: ansible  
  options:
    vvv: true
    diff: true
  env:
    FOO: BAR

scenario

The name and description of scenario sequences.
You can change the default action matrix of any command by adding the key _sequence and defining the necessary step list as its value.
For example, we want to change the action sequence when running the command to execute the playbook: molecule converge

# изначально:
# - dependency
# - create
# - prepare
# - converge
scenario:
  name: default
  converge_sequence:
    - create
    - converge

verifier

Setting up the test framework and its linter. By default, the linter used is testinfra and flake8. Possible options are similar to those mentioned above:

verifier:
  name: testinfra
  additional_files_or_dirs:
    - ../path/to/test_1.py
    - ../path/to/test_2.py
    - ../path/to/directory/*
  options:
    n: 1
  enabled: False
  env:
    FOO: bar
  lint:
    name: flake8
    options:
      benchmark: True
    enabled: False
    env:
      FOO: bar

Let’s return to our role. Edit the file tasks/main.yml to look like this:

---
- name: Install nginx
  apt:
    name: nginx
    state: present

- name: Start nginx
  service:
    name: nginx
    state: started

And let's add tests in molecule/default/tests/test_default.py

def test_nginx_is_installed(host):
    nginx = host.package("nginx")
    assert nginx.is_installed

def test_nginx_running_and_enabled(host):
    nginx = host.service("nginx")
    assert nginx.is_running
    assert nginx.is_enabled

def test_nginx_config(host):
    host.run("nginx -t")

Done, just need to run (from the role root, I remind you):

> molecule test

Long output under the spoiler:

--> Validating schema /nginx/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix

└── default
    ├── lint
    ├── destroy
    ├── dependency
    ├── syntax
    ├── create
    ├── prepare
    ├── converge
    ├── idempotence
    ├── side_effect
    ├── verify
    └── destroy

--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /nginx/... 
Lint completed successfully.
--> Executing Flake8 on files found in /nginx/molecule/default/tests/... 
Lint completed successfully.
--> Executing Ansible Lint on /nginx/molecule/default/playbook.yml...
Lint completed successfully.
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) deletion to complete] *******************************
    ok: [localhost] => (item=None)
    ok: [localhost]

    TASK [Delete docker network(s)] ************************************************

    PLAY RECAP *********************************************************************
    localhost                  : ok=2    changed=1    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.
--> Scenario: 'default'
--> Action: 'syntax'

    playbook: /nginx/molecule/default/playbook.yml

--> Scenario: 'default'
--> Action: 'create'

    PLAY [Create] ******************************************************************

    TASK [Log into a Docker registry] **********************************************
    skipping: [localhost] => (item=None)

    TASK [Create Dockerfiles from image names] *************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Discover local Docker images] ********************************************
    ok: [localhost] => (item=None)
    ok: [localhost]

    TASK [Build an Ansible compatible image] ***************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Create docker network(s)] ************************************************

    TASK [Create molecule instance(s)] *********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) creation to complete] *******************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    PLAY RECAP *********************************************************************
    localhost                  : ok=5    changed=4    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'prepare'
Skipping, prepare playbook not configured.
--> Scenario: 'default'
--> Action: 'converge'

    PLAY [Converge] ****************************************************************

    TASK [Gathering Facts] *********************************************************
    ok: [instance]

    TASK [nginx : Install nginx] ***************************************************
    changed: [instance]

    TASK [nginx : Start nginx] *****************************************************
    changed: [instance]

    PLAY RECAP *********************************************************************
    instance                   : ok=3    changed=2    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'idempotence'
Idempotence completed successfully.
--> Scenario: 'default'
--> Action: 'side_effect'
Skipping, side effect playbook not configured.
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /nginx/molecule/default/tests/... 
    ============================= test session starts ==============================
    platform darwin -- Python 2.7.15, pytest-4.3.0, py-1.8.0, pluggy-0.9.0
    rootdir: /nginx/molecule/default, inifile:
    plugins: testinfra-1.16.0
collected 4 items

    tests/test_default.py ....                                               [100%]

    ========================== 4 passed in 27.23 seconds ===========================
Verifier completed successfully.
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) deletion to complete] *******************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Delete docker network(s)] ************************************************

    PLAY RECAP *********************************************************************
    localhost                  : ok=2    changed=2    unreachable=0    failed=0

Our simple role tested without issues.
It's important to remember that if any problems arose during operation molecule test, if you haven't altered the standard sequence, Molecule will remove the instance.

The following commands are useful for debugging:

> molecule --debug  # debug info. By default, Molecule hides logs.
> molecule converge          # Keeps the instance after running the test role.
> molecule login             # Log into the created instance.
> molecule --help            # Full list of commands.

Existing role

Adding a new scenario to an existing role occurs from the role directory using the following commands:

# полный список доступных параметров
> molecule init scenarion --help
# создание нового сценария
> molecule init scenario -r <role_name> -s <scenario_name>

If this is the first scenario in the role, the parameter -s can be omitted, as a scenario will be created. default.

Conclusion

As you can see, Molecule isn't very complicated, and by using your own templates, you can reduce the deployment of a new scenario to just editing variables in the instance creation and deletion playbooks. Molecule integrates seamlessly with CI systems, allowing for faster development by reducing manual testing time for playbooks.

Thank you for your attention. If you have experience testing ansible roles that is not related to Molecule, please share it in the comments!

Source: habr.com