RHEL 8 Beta offers developers a lot of new features, listing which could take pages, however, learning new things is always better in practice, so below we offer a practice of actually creating an application infrastructure based on Red Hat Enterprise Linux 8 Beta.
We will take Python, a programming language popular among developers, as a basis, a combination of Django and PostgreSQL, a fairly common bundle for creating applications, and configure RHEL 8 Beta to work with them. Then we'll add a couple more (unclassified) ingredients.
The test environment will change as it is interesting to explore the possibilities of automation, work with containers and try environments with multiple servers. To get started with a new project, you can start by creating a small, simple prototype by hand so you can see exactly what needs to happen and how the interaction is carried out, and then move on to automation and creating more complex configurations. Today is a story about the creation of such a prototype.
Let's start by deploying the RHEL 8 Beta VM image. You can install a virtual machine from scratch, or use the KVM guest image available with a Beta subscription. When using a guest image, you will need to set up a virtual CD that will contain metadata and user data for cloud provisioning (cloud-init). You don't need to do anything special with the disk structure or available packages, any configuration will do.
Let's look at the whole process in more detail.
Installing Django
With the latest version of Django, you will need a virtual environment (virtualenv) with Python 3.5 or later. You can see in the Beta notes that Python 3.6 is available, let's check if this is indeed the case:
[cloud-user@8beta1 ~]$ python
-bash: python: command not found
[cloud-user@8beta1 ~]$ python3
-bash: python3: command not found
Red Hat actively uses Python as a system toolkit in RHEL, so why is this the result?
The fact is that many developers using Python are still thinking about switching from Python 2 to Python 2, while Python 3 itself is under active development, and more and more new versions are constantly appearing. Therefore, in order to meet the need for stable system tools, while also offering users access to various new versions of Python, the system Python has been ported to a new package, and both Python 2.7 and 3.6 can be installed. More information about the changes and why this was done can be gleaned from a post in (Langdon White).
So, to get a working Python, you need to install only two packages, while python3-pip will be pulled up as a dependency.
sudo yum install python36 python3-virtualenv
Why not use direct module calls like Langdon suggests and install pip3? With automation in mind, it is known that Ansible will require pip to be installed, since the pip module does not support virtual environments (virtualenvs) with a custom pip executable.
With a working python3 interpreter at our disposal, we can continue with the Django installation process and have a working system along with our other components. There are many implementation options on the web. This is one version, but users can use their own processes.
The PostgreSQL and Nginx versions available in RHEL 8 will be installed by default using Yum.
sudo yum install nginx postgresql-server
PostgreSQL will require psycopg2, but it only needs to be available in a virtualenv environment, so we will install it with pip3 along with Django and Gunicorn. But first we need to set up the virtualenv.
There is always a lot of debate about choosing the right place to install Django projects, but when in doubt, you can always refer to the Linux Filesystem Hierarchy Standard. Specifically, the FHS says that /srv is used to: βstorage host-specific data - data that the system produces, such as web server data and scripts, data stored on FTP servers, and control system repositories versions (introduced in FHS-2.3 in 2004)."
This is just our case, so we put everything we need in / srv, which is owned by our application user (cloud-user).
sudo mkdir /srv/djangoapp
sudo chown cloud-user:cloud-user /srv/djangoapp
cd /srv/djangoapp
virtualenv django
source django/bin/activate
pip3 install django gunicorn psycopg2
./django-admin startproject djangoapp /srv/djangoapp
Setting up PostgreSQL and Django is easy: create a database, create a user, set permissions. One thing to keep in mind when installing PostgreSQL initially is the postgresql-setup script, which is installed with the postgresql-server package. This script helps you perform basic database cluster administration tasks, such as cluster initialization or the upgrade process. To set up a new PostgreSQL instance on a RHEL system, we need to run the command:
sudo /usr/bin/postgresql-setup -initdb
After that, you can start PostgreSQL using systemd, create a database, and set up a project in Django. Remember to restart PostgreSQL after making changes to the client authentication configuration file (usually pg_hba.conf) to configure password storage for the application user. If you run into other issues, make sure to change the IPv4 and IPv6 settings in the pg_hba.conf file.
systemctl enable -now postgresql
sudo -u postgres psql
postgres=# create database djangoapp;
postgres=# create user djangouser with password 'qwer4321';
postgres=# alter role djangouser set client_encoding to 'utf8';
postgres=# alter role djangouser set default_transaction_isolation to 'read committed';
postgres=# alter role djangouser set timezone to 'utc';
postgres=# grant all on DATABASE djangoapp to djangouser;
postgres=# q
In the /var/lib/pgsql/data/pg_hba.conf file:
# IPv4 local connections:
host all all 0.0.0.0/0 md5
# IPv6 local connections:
host all all ::1/128 md5
In /srv/djangoapp/settings.py file:
# Database
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': '{{ db_name }}',
'USER': '{{ db_user }}',
'PASSWORD': '{{ db_password }}',
'HOST': '{{ db_host }}',
}
}
After configuring the settings.py file in the project and setting up the database configuration, you can start the development server to make sure everything works. Once the development server is up and running, it's a good idea to create an admin user to test the database connection.
./manage.py runserver 0.0.0.0:8000
./manage.py createsuperuser
WSGI? Wai?
The development server is useful for testing, but you must configure the appropriate server and proxy for the Web Server Gateway Interface (WSGI) to run your application. There are several common bundles, such as Apache HTTPD with uWSGI or Nginx with Gunicorn.
The job of the Web Server Gateway Interface is to forward requests from the web server to the Python web framework. WSGI is a legacy of a terrible past when CGI mechanisms were in use, and today WSGI is the de facto standard, regardless of the web server or Python framework used. But despite its wide distribution, there are still many nuances when working with these frameworks, and many options to choose from. In this case, we will try to establish interaction between Gunicorn and Nginx via a socket.
Since both of these components are installed on the same server, let's try using a UNIX socket instead of a network socket. Since communication needs a socket anyway, let's try to take it a step further and set up socket activation for Gunicorn via systemd.
The process of creating socket activated services is quite simple. First, a unit file is created that contains a ListenStream directive pointing to the point at which the UNIX socket will be created, then a unit file for the service, in which the Requires directive will point to the socket unit file. Then, in the service unit file, all that remains is to call Gunicorn from the virtual environment and create a WSGI binding for the UNIX socket and Django application.
Here are some examples of unit files that can be taken as a basis. First we set up the socket.
[Unit]
Description=Gunicorn WSGI socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
Now we need to configure the Gunicorn daemon.
[Unit]
Description=Gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=cloud-user
Group=cloud-user
WorkingDirectory=/srv/djangoapp
ExecStart=/srv/djangoapp/django/bin/gunicorn
βaccess-logfile -
βworkers 3
βbind unix:gunicorn.sock djangoapp.wsgi
[Install]
WantedBy=multi-user.target
For Nginx, it's as simple as creating proxy configuration files and setting up a static content directory if you're using one. On RHEL, the Nginx configuration files are /etc/nginx/conf.d. You can copy the following example there to the /etc/nginx/conf.d/default.conf file and start the service. Make sure to set server_name to match your host name.
server {
listen 80;
server_name 8beta1.example.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /srv/djangoapp;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
Start socketing Gunicorn and Nginx with systemd and you're good to go.
Bad Gateway Error?
If you enter the address into a browser, you will most likely get a 502 Bad Gateway error. It can be caused by misconfigured UNIX socket permissions, or more complex access control issues in SELinux.
In the nginx error log, you can find a line like this:
2018/12/18 15:38:03 [crit] 12734#0: *3 connect() to unix:/run/gunicorn.sock failed (13: Permission denied) while connecting to upstream, client: 192.168.122.1, server: 8beta1.example.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "8beta1.example.com"
If we test Gunicorn directly, we get an empty answer.
curl βunix-socket /run/gunicorn.sock 8beta1.example.com
Let's see why this is happening. If you open the log, then most likely we will see that the problem is related to SELinux. Since we have a daemon running for which no policy has been created, it is marked as init_t. Let's test this theory in practice.
sudo setenforce 0
All this can cause criticism and bloody tears, but this is just debugging a prototype. Let's turn off the check just to make sure that this is the problem, after which we will return everything back to its place.
By refreshing the page in the browser or by re-running our curl command, we can see the Django test page.
So, after making sure everything works and there are no more permission issues, we re-enable SELinux.
sudo setenforce 1
I won't talk about audit2allow and creating alert-based policies with sepolgen here, since there's no real Django application at the moment, there's also no complete map of what Gunicorn might want to access and deny. Therefore, it is necessary to keep SELinux running to protect the system, while at the same time allowing the application to run and leave messages in the audit log so that real policy can then be created from them.
Specifying Permissive Domains
Not everyone has heard of allowed domains in SELinux, but they are nothing new. Many even worked with them without realizing it. When a policy is created based on audit messages, the generated policy represents the allowed domain. Let's try to create a simple permissive policy.
To create a specific allowed domain for Gunicorn, you need some kind of policy, and you also need to flag the appropriate files. In addition, tools are needed to assemble new policies.
sudo yum install selinux-policy-devel
The Allowed Domains mechanism is a great tool for identifying problems, especially when it comes to a custom application or applications that come without policies already created. In this case, the allowed domain policy for Gunicorn will be as simple as possible - we will declare a main type (gunicorn_t), we will declare a type that we will use to mark multiple executable files (gunicorn_exec_t), and then we will configure a transition (transition) for system to correctly mark running processes . The last line sets the policy as enabled by default when it is loaded.
gunicorn.te:
policy_module(gunicorn, 1.0)
type gunicorn_t;
type gunicorn_exec_t;
init_daemon_domain(gunicorn_t, gunicorn_exec_t)
permissive gunicorn_t;
You can compile this policy file and add it to your system.
make -f /usr/share/selinux/devel/Makefile
sudo semodule -i gunicorn.pp
sudo semanage permissive -a gunicorn_t
sudo semodule -l | grep permissive
Let's see if SELinux is blocking something else besides what our unknown daemon is accessing.
sudo ausearch -m AVC
type=AVC msg=audit(1545315977.237:1273): avc: denied { write } for pid=19400 comm="nginx" name="gunicorn.sock" dev="tmpfs" ino=52977 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file permissive=0
SELinux prevents Nginx from writing data to the UNIX socket used by Gunicorn. Usually in such cases, they begin to change policies, but there are other tasks ahead. You can also change the domain settings from a restriction domain to an allow domain. Now let's move httpd_t to the permission domain. This will give Nginx the necessary access so we can continue with further debugging work.
sudo semanage permissive -a httpd_t
So, once you've managed to keep SELinux protected (in fact, you shouldn't leave your SELinux project in restricted mode) and the permission domains are loaded, you need to figure out what exactly needs to be marked as gunicorn_exec_t to get everything working properly again. Let's try to access the website to see the new messages about access restrictions.
sudo ausearch -m AVC -c gunicorn
You can see a lot of messages containing 'comm="gunicorn"' that perform various actions on files in /srv/djangoapp, so this is obviously one of the commands worth flagging.
But in addition, a message like this appears:
type=AVC msg=audit(1545320700.070:1542): avc: denied { execute } for pid=20704 comm="(gunicorn)" name="python3.6" dev="vda3" ino=8515706 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0
If you look at the status of the gunicorn service or run the ps command, there won't be any processes running. It looks like gunicorn is trying to access the Python interpreter in our virtualenv, perhaps to run workers scripts. So now let's mark these two executables and see if our Django test page can open.
chcon -t gunicorn_exec_t /srv/djangoapp/django/bin/gunicorn /srv/djangoapp/django/bin/python3.6
The gunicorn service will need to be restarted before the new label can be selected. You can restart it immediately or stop the service and let the socket start it when you open the site in the browser. Verify that the processes have received the correct labels using ps.
ps -efZ | grep gunicorn
Don't forget to create a normal SELinux policy afterwards!
Looking at the AVC messages now, the last message contains permissive=1 for everything related to the application and permissive=0 for the rest of the system. If you understand exactly what kind of access a real application needs, you can quickly find the best way to solve such problems. But until then, it's best to keep the system secure and get a clear and usable audit of the Django project.
sudo ausearch -m AVC
Happened!
A working Django project appeared with a frontend on Nginx and Gunicorn WSGI. We have configured Python 3 and PostgreSQL 10 from the RHEL 8 Beta repositories. You can now move on and create (or just deploy) Django applications or explore other available tools in RHEL 8 Beta to automate the process of customizing, improving performance, or even containerizing that configuration.
Source: habr.com