Not so long ago, I needed to write several ansible playbooks to prepare the server for deploying a rails application. And, surprisingly, I did not find a simple step-by-step manual. I didn’t want to copy someone else’s playbook without understanding what was happening, and as a result I had to read the documentation, collecting everything on my own. Perhaps I can help someone speed up this process with the help of this article.
The first thing to understand is that ansible provides you with a convenient interface to perform a predefined list of actions on a remote server(s) via SSH. There is no magic here, you cannot install a plugin and get a zero downtime deployment of your application with docker, monitoring and other goodies out of the box. In order to write a playbook you need to know exactly what you want to do and how to do it. Therefore, I am not satisfied with ready-made playbooks from github, or articles like: “Copy and run, it will work.”
What do we need?
As I said, in order to write a playbook you need to know what you want to do and how to do it. Let's decide what we need. For a Rails application, we need several system packages: nginx, postgresql (redis, etc). In addition, we need ruby of a certain version. It is best to install it through rbenv (rvm, asdf...). Running all this as a root user is always a bad idea, so you need to create a separate user and set up permissions for him. After that, you need to upload our code to the server, copy the configs for nginx, postgres, etc and start all these services.
As a result, the sequence of actions is as follows:
- Login under root
- install system packages
- create a new user, configure rights, ssh key
- configure system packages (nginx etc) and run them
- Create a user in the database (you can immediately create a database)
- Logging in as a new user
- Installing rbenv and ruby
- Installing the bundler
- Uploading the application code
- Starting the Puma Server
Moreover, the last steps can be done using capistrano, at least it can copy the code to release directories out of the box, switch the release with a symlink upon successful deployment, copy configs from the shared directory, restart puma, etc. All this can be done with Ansible, but why?
File structure
Ansible has a strict for all your files, so it's best to keep it all in a separate directory. And it is not so important whether it will be in the rails application itself, or separately. You can store files in a separate git repository. Personally, I found it most convenient to create an ansible directory in the /config directory of the rails application and store everything in one repository.
simple playbook
A playbook is a yml file that describes, using a special syntax, what and how ansible should do. Let's create the first playbook that does nothing:
---
- name: Simple playbook
hosts: allHere we just say that our playbook is called Simple Playbook and that its content must run for all hosts. We can store it in /ansible directory named playbook.yml and try to run:
ansible-playbook ./playbook.yml
PLAY [Simple Playbook] ************************************************************************************************************************************
skipping: no hosts matchedAnsible says it doesn't know any hosts that match the all list. They must be listed in a special .
Let's create it in the same ansible directory:
123.123.123.123This is how we simply specify the host (ideally, the host of our VPS for tests, or you can register localhost) and save it under the name inventory.
You can try running ansible with an inventory file:
ansible-playbook ./playbook.yml -i inventory
PLAY [Simple Playbook] ************************************************************************************************************************************
TASK [Gathering Facts] ************************************************************************************************************************************
PLAY RECAP ************************************************************************************************************************************If you have ssh access to the specified host then ansible will connect and collect information about the remote system. (default TASK [Gathering Facts] ) after which it will give a summary of the execution (PLAY RECAP).
By default, the connection uses the username under which you are logged into the system. Most likely, it will not be on the host. In the playbook file, you can specify which user to use to connect using the remote_user directive. Also, information about a remote system may often be unnecessary for you and you should not waste time collecting it. This task can also be disabled:
---
- name: Simple playbook
hosts: all
remote_user: root
become: true
gather_facts: noTry running the playbook again and make sure the connection is working. (If you specified the root user, then you also need to specify the become: true directive to get elevated rights. As it is written in the documentation: become set to ‘true’/’yes’ to activate privilege escalation. although it is not entirely clear why).
Perhaps you will get an error caused by the fact that ansible cannot determine the python interpreter, then you can specify it manually:
ansible_python_interpreter: /usr/bin/python3 where you have python, you can find out with the command whereis python.
Installing system packages
The standard distribution of Ansible includes many modules for working with various system packages, so we do not have to write bash scripts for any reason. Now we need one of these modules to update the system and install system packages. I have Ubuntu Linux on my VPS, so I use apt-get и . If you use a different operating system, then you may need a different module (remember, I said at the beginning that you need to know in advance what and how we will do). However, the syntax is likely to be similar.
Let's complete our playbook with the first tasks:
---
- name: Simple playbook
hosts: all
remote_user: root
become: true
gather_facts: no
tasks:
- name: Update system
apt: update_cache=yes
- name: Install system dependencies
apt:
name: git,nginx,redis,postgresql,postgresql-contrib
state: presentTask is just the task that ansible will perform on remote servers. We give the task a name to track its execution in the log. And we describe, using the syntax of a particular module, what it needs to do. In this case apt: update_cache=yes - says to update the system packages using the apt module. The second command is somewhat more complicated. We pass a list of packages to the apt module, and say that they state should become present, that is, we say install these packages. Similarly, we can tell them to delete, or update, by simply changing state. Note that for rails to work with postgresql, we need the postgresql-contrib package, which we are currently installing. Again, you need to know and do this, ansible by itself will not do this.
Try running the playbook again and check that the packages install.
Creation of new users.
To work with users, Ansible also has a module - user. Let's add one more task (I hid the already known parts of the playbook behind the comments so as not to copy it entirely every time):
---
- name: Simple playbook
# ...
tasks:
# ...
- name: Add a new user
user:
name: my_user
shell: /bin/bash
password: "{{ 123qweasd | password_hash('sha512') }}"We create a new user, set a shell and a password for it. And then we run into several problems. What if usernames should be different for different hosts? Yes, and storing the password in clear text in the playbook is a very bad idea. To begin with, we will take out the username and password in variables, and towards the end of the article I will show how to encrypt the password.
---
- name: Simple playbook
# ...
tasks:
# ...
- name: Add a new user
user:
name: "{{ user }}"
shell: /bin/bash
password: "{{ user_password | password_hash('sha512') }}"Variables are set using double curly braces in playbooks.
We will specify the values of the variables in the inventory file:
123.123.123.123
[all:vars]
user=my_user
user_password=123qweasdPay attention to the directive [all:vars] - she says that the next block of text is variables (vars) and they are applicable to all hosts (all).
Also interesting design "{{ user_password | password_hash('sha512') }}". The point is that ansible does not set the user via user_add like you would do it manually. And it saves all the data directly, which is why we also need to convert the password to a hash in advance, which this command does.
Let's add our user to the sudo group. However, before that, you need to make sure that such a group exists because no one will do this for us:
---
- name: Simple playbook
# ...
tasks:
# ...
- name: Ensure a 'sudo' group
group:
name: sudo
state: present
- name: Add a new user
user:
name: "{{ user }}"
shell: /bin/bash
password: "{{ user_password | password_hash('sha512') }}"
groups: "sudo"It's quite simple, we also have a group module for creating groups, with a syntax very similar to apt. After that, it is enough to register this group to the user (groups: "sudo").
It is also useful to add an ssh key to this user so that we can log in without a password:
---
- name: Simple playbook
# ...
tasks:
# ...
- name: Ensure a 'sudo' group
group:
name: sudo
state: present
- name: Add a new user
user:
name: "{{ user }}"
shell: /bin/bash
password: "{{ user_password | password_hash('sha512') }}"
groups: "sudo"
- name: Deploy SSH Key
authorized_key:
user: "{{ user }}"
key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
state: presentIn this case, the construction "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" - it copies the contents of the id_rsa.pub file (your name may differ), that is, the public part of the ssh key and uploads it to the list of authorized keys for the user on the server.
Roles
All three tasks for creating use can be easily assigned to the same task group, and it would be nice to keep this group separate from the main playbook so that it does not grow too much. For this, ansible has .
According to the file structure indicated at the very beginning, the roles must be placed in a separate roles directory, for each role there is a separate directory with the same name, inside the tasks, files, templates, etc directory
Let's create a file structure: ./ansible/roles/user/tasks/main.yml (main is the main file that will be loaded and executed when the role is connected to the playbook, other role files can be connected in it). Now you can transfer all tasks related to the user to this file:
# Create user and add him to groups
- name: Ensure a 'sudo' group
group:
name: sudo
state: present
- name: Add a new user
user:
name: "{{ user }}"
shell: /bin/bash
password: "{{ user_password | password_hash('sha512') }}"
groups: "sudo"
- name: Deploy SSH Key
authorized_key:
user: "{{ user }}"
key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
state: presentIn the main playbook, you must specify to use the user role:
---
- name: Simple playbook
hosts: all
remote_user: root
gather_facts: no
tasks:
- name: Update system
apt: update_cache=yes
- name: Install system dependencies
apt:
name: git,nginx,redis,postgresql,postgresql-contrib
state: present
roles:
- userAlso, it may make sense to update the system before all other tasks, for this you can rename the block tasks in which they are defined in pre_tasks.
Setting up nginx
We should already have Nginx installed, we need to configure it and run it. Let's do it right away in the role. Create a file structure:
- ansible
- roles
- nginx
- files
- tasks
- main.yml
- templatesNow we need files and templates. The difference between them is that ansible copies files directly, as is. And templates must have the extension j2 and you can use the values of variables in them using the same double curly braces.
Let's enable nginx in main.yml file. For this we have the systemd module:
# Copy nginx configs and start it
- name: enable service nginx and start
systemd:
name: nginx
state: started
enabled: yesHere we not only say that nginx must be started (that is, we start it), but we immediately say that it must be enabled.
Now let's copy the configuration files:
# Copy nginx configs and start it
- name: enable service nginx and start
systemd:
name: nginx
state: started
enabled: yes
- name: Copy the nginx.conf
copy:
src: nginx.conf
dest: /etc/nginx/nginx.conf
owner: root
group: root
mode: '0644'
backup: yes
- name: Copy template my_app.conf
template:
src: my_app_conf.j2
dest: /etc/nginx/sites-available/my_app.conf
owner: root
group: root
mode: '0644'We create the main nginx configuration file (you can take it directly from the server, or write it yourself). And also the config file for our application in the sites_available directory (this is not required but useful). In the first case, we use the copy module to copy files (the file must be in /ansible/roles/nginx/files/nginx.conf). In the second, we copy the template, substituting the values of the variables. The template must be in /ansible/roles/nginx/templates/my_app.j2). And it might look something like this:
upstream {{ app_name }} {
server unix:{{ app_path }}/shared/tmp/sockets/puma.sock;
}
server {
listen 80;
server_name {{ server_name }} {{ inventory_hostname }};
root {{ app_path }}/current/public;
try_files $uri/index.html $uri.html $uri @{{ app_name }};
....
}Pay attention to inserts {{ app_name }}, {{ app_path }}, {{ server_name }}, {{ inventory_hostname }} are all variables whose values ansible will substitute into the template before copying. This is useful if you use a playbook for different groups of hosts. For example, we can add to our inventory file:
[production]
123.123.123.123
[staging]
231.231.231.231
[all:vars]
user=my_user
user_password=123qweasd
[production:vars]
server_name=production
app_path=/home/www/my_app
app_name=my_app
[staging:vars]
server_name=staging
app_path=/home/www/my_stage
app_name=my_stage_appIf we run our playbook now, it will execute the specified tasks for both hosts. But at the same time, for the staging host, the variables will be different from the production ones, and not only in roles and playbooks, but also in nginx configs. {{ inventory_hostname }} do not need to be specified in the inventory file - this and the host for which the playbook is currently running is stored there.
If you want to have an inventory file for multiple hosts but only run for one group, you can do so with the following command:
ansible-playbook -i inventory ./playbook.yml -l "staging"another option is to have separate inventory files for different groups. Or you can combine the two approaches if you have many different hosts.
Let's get back to setting up nginx. After copying the configuration files, we need to create a symlink in sitest_enabled to my_app.conf from sites_available. And restart nginx.
... # old code in mail.yml
- name: Create symlink to sites-enabled
file:
src: /etc/nginx/sites-available/my_app.conf
dest: /etc/nginx/sites-enabled/my_app.conf
state: link
- name: restart nginx
service:
name: nginx
state: restartedEverything is simple here - again ansible modules with a fairly standard syntax. But there is one moment. Restarting nginx every time doesn't make sense. You noticed that we do not write commands like: “do this like this”, the syntax looks more like “this should have such a state”. And most often this is how ansible works. If the group already exists, or the system package is already installed, then ansible will check this and skip the task. Also, files will not be copied if they completely match what is already on the server. We can take advantage of this and restart nginx only if the configuration files have changed. There is a register directive for this:
# Copy nginx configs and start it
- name: enable service nginx and start
systemd:
name: nginx
state: started
enabled: yes
- name: Copy the nginx.conf
copy:
src: nginx.conf
dest: /etc/nginx/nginx.conf
owner: root
group: root
mode: '0644'
backup: yes
register: restart_nginx
- name: Copy template my_app.conf
template:
src: my_app_conf.j2
dest: /etc/nginx/sites-available/my_app.conf
owner: root
group: root
mode: '0644'
register: restart_nginx
- name: Create symlink to sites-enabled
file:
src: /etc/nginx/sites-available/my_app.conf
dest: /etc/nginx/sites-enabled/my_app.conf
state: link
- name: restart nginx
service:
name: nginx
state: restarted
when: restart_nginx.changedIf one of the configuration files changes, then a copy will be made and a variable will be registered restart_nginx. And only if this variable has been registered, the service will be restarted.
And, of course, you need to add the nginx role to the main playbook.
postgresql setup
We need to enable postgresql with systemd just like we did with nginx, and also create a user that we will use to access the database and the database itself.
Let's create a role /ansible/roles/postgresql/tasks/main.yml:
# Create user in postgresql
- name: enable postgresql and start
systemd:
name: postgresql
state: started
enabled: yes
- name: Create database user
become_user: postgres
postgresql_user:
name: "{{ db_user }}"
password: "{{ db_password }}"
role_attr_flags: SUPERUSER
- name: Create database
become_user: postgres
postgresql_db:
name: "{{ db_name }}"
encoding: UTF-8
owner: "{{ db_user }}"I will not describe how to add variables to inventory, this has already been done many times, as well as the syntax of the postgresql_db and postgresql_user modules. More information can be found in the documentation. Here is the most interesting directive become_user: postgres. The fact is that by default, only the postgres user has access to the postgresql database and only locally. This directive allows us to execute commands on behalf of this user (if we have access, of course).
Also, you may need to add a line to pg_hba.conf to allow a new user access to the database. This can be done in the same way as we changed the nginx config.
And of course, you need to add the postgresql role to the main playbook.
Installing ruby via rbenv
Ansible does not have modules for working with rbenv, but it is installed by cloning the git repository. Therefore, this task becomes the most non-standard. Let's create a role for her /ansible/roles/ruby_rbenv/main.yml and start filling it in:
# Install rbenv and ruby
- name: Install rbenv
become_user: "{{ user }}"
git: repo=https://github.com/rbenv/rbenv.git dest=~/.rbenvWe again use the become_user directive to work from under the user we created for this purpose. Because rbenv is installed in its home directory, not globally. And we also use the git module to clone the repository by specifying repo and dest.
Next, we need to add rbenv init to bashrc and add rbenv to PATH in the same place. For this we have the lineinfile module:
- name: Add rbenv to PATH
become_user: "{{ user }}"
lineinfile:
path: ~/.bashrc
state: present
line: 'export PATH="${HOME}/.rbenv/bin:${PATH}"'
- name: Add rbenv init to bashrc
become_user: "{{ user }}"
lineinfile:
path: ~/.bashrc
state: present
line: 'eval "$(rbenv init -)"'Then you need to install ruby_build:
- name: Install ruby-build
become_user: "{{ user }}"
git: repo=https://github.com/rbenv/ruby-build.git dest=~/.rbenv/plugins/ruby-buildAnd finally install ruby. This is done through rbenv, i.e. just bash with the command:
- name: Install ruby
become_user: "{{ user }}"
shell: |
export PATH="${HOME}/.rbenv/bin:${PATH}"
eval "$(rbenv init -)"
rbenv install {{ ruby_version }}
args:
executable: /bin/bashWe say which command to execute and with what. However, here we will stumble upon the fact that ansible does not run the code contained in bashrc before running commands. This means that rbenv will have to be defined directly in the same script.
The next issue is that the shell command is stateless from ansible's point of view. That is, there will be no automatic check whether this version of ruby is installed or not. We can do it ourselves:
- name: Install ruby
become_user: "{{ user }}"
shell: |
export PATH="${HOME}/.rbenv/bin:${PATH}"
eval "$(rbenv init -)"
if ! rbenv versions | grep -q {{ ruby_version }}
then rbenv install {{ ruby_version }} && rbenv global {{ ruby_version }}
fi
args:
executable: /bin/bashAnd it remains to install the bundler:
- name: Install bundler
become_user: "{{ user }}"
shell: |
export PATH="${HOME}/.rbenv/bin:${PATH}"
eval "$(rbenv init -)"
gem install bundlerAnd again add our ruby_rbenv role to the main playbook.
shared files.
In general, this setting could be completed. Then it remains to run capistrano and it will copy the code itself, create the necessary directories and launch the application (if everything is configured correctly). However, capistrano often needs additional configuration files, such as database.yml or .env They can be copied just like files and templates for nginx. There is only one subtlety. Before copying files, you need to create a directory structure for them, something like this:
# Copy shared files for deploy
- name: Ensure shared dir
become_user: "{{ user }}"
file:
path: "{{ app_path }}/shared/config"
state: directorywe specify only one directory and ansible will automatically create the parent if needed.
Ansible Vault
We have already stumbled upon the fact that variables can contain secret data such as the user's password. If you have created .env application file, and database.yml then there must be even more of such critical data. It would be nice to hide them from prying eyes. For this, it is used .
Create a file for variables /ansible/vars/all.yml (here you can create different files for different groups of hosts, just like in the inventory file: production.yml, staging.yml, etc).
All variables must be transferred to this file, which must be encrypted using the standard yml syntax:
# System vars
user_password: 123qweasd
db_password: 123qweasd
# ENV vars
aws_access_key_id: xxxxx
aws_secret_access_key: xxxxxx
aws_bucket: bucket_name
rails_secret_key_base: very_secret_key_baseAfter that, this file can be encrypted with the command:
ansible-vault encrypt ./vars/all.ymlNaturally, when encrypting, you will need to set a password for decryption. You can see what will be inside the file after calling this command.
By means of ansible-vault decrypt the file can be decrypted, modified and then encrypted again.
You don't need to decrypt the file to work. You store it encrypted and run the playbook with the argument --ask-vault-pass. Ansible will ask for the password, get the variables, and run the tasks. All data will remain encrypted.
The full command for multiple hostgroups and ansible vault would look something like this:
ansible-playbook -i inventory ./playbook.yml -l "staging" --ask-vault-passAnd I won’t give you the full text of the playbooks and roles, write it yourself. Because ansible is such a thing - if you don’t understand what needs to be done, then he won’t do it to you either.
Source: habr.com