Elastic Stack is a well-known tool in the SIEM systems market (actually, not only them). It can collect a lot of different-sized data, both sensitive and not very sensitive. It is not entirely correct if access to the Elastic Stack elements themselves is not protected. By default, all Elastic boxed elements (Elasticsearch, Logstash, Kibana, and Beats collectors) operate on open protocols. And in Kibana itself, authentication is disabled. All these interactions can be secured, and in this article we will tell you how to do it. For convenience, the story was divided into 3 semantic blocks:

• Role model of data access
• Data security inside an Elasticsearch cluster
• Data security outside of an Elasticsearch cluster

Details under the cut.

Role model of data access

If you install Elasticsearch and tune it in no way, access to all indexes will be open to everyone. Well, or those who can use curl. To avoid this, Elasticsearch has a role model that is available starting with a Basic level subscription (which is free). Schematically it looks like this:

What's in the picture

• Users are all who can log in using credentials.
• A role is a set of rights.
• Rights are a set of privileges.
• Privileges are permissions to write, read, delete, etc. (Full list of privileges)
• Resources are indexes, documents, fields, users, and other storage entities (the role model for some resources is only available in paid subscriptions).

Elasticsearch has by default boxed usersto which are attached box roles. After enabling security settings, you can start using them immediately.

To enable security in the Elasticsearch settings, you need to add it to the configuration file (by default this is elasticsearch/config/elasticsearch.yml) new line:

xpack.security.enabled: true

After changing the configuration file, start or restart Elasticsearch for the changes to take effect. The next step is to assign passwords to boxed users. Let's do this interactively with the command below:

[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

We check:

[elastic@node1 ~]$ curl -u elastic 'node1:9200/_cat/nodes?pretty'
Enter host password for user 'elastic':
192.168.0.2 23 46 14 0.28 0.32 0.18 dim * node1

You can slap yourself on the shoulder - the settings on the Elasticsearch side are done. Now it's the turn to set up Kibana. If you run it now, errors will fall, so it is important to create a keystore. This is done in two commands (user kibana and the password entered in the step of creating passwords in Elasticsearch):

[elastic@node1 ~]$ ./kibana/bin/kibana-keystore add elasticsearch.username
[elastic@node1 ~]$ ./kibana/bin/kibana-keystore add elasticsearch.password

If everything is correct, Kibana will start asking for a username and password. In a Basic subscription, a role model based on internal users is available. Starting with Gold, you can connect external authentication systems - LDAP, PKI, Active Directory and Single sign-on systems.

Access rights to objects inside Elasticsearch can also be limited. True, to do the same for documents or fields, you will need a paid subscription (this luxury starts with the Platinum level). These settings are available in the Kibana interface or via Security API. You can check through the already familiar Dev Tools menu:

Create a Role

PUT /_security/role/ruslan_i_ludmila_role
{
  "cluster": [],
  "indices": [
    {
      "names": [ "ruslan_i_ludmila" ],
      "privileges": ["read", "view_index_metadata"]
    }
  ]
}

User creation

POST /_security/user/pushkin
{
  "password" : "nataliaonelove",
  "roles" : [ "ruslan_i_ludmila_role", "kibana_user" ],
  "full_name" : "Alexander Pushkin",
  "email" : "[email protected]",
  "metadata" : {
    "hometown" : "Saint-Petersburg"
  }
}

Data security inside an Elasticsearch cluster

When Elasticsearch runs in a cluster (which is common), security settings within the cluster become important. For secure communication between nodes, Elasticsearch uses the TLS protocol. To set up secure communication between them, you need a certificate. We generate a certificate and a private key in PEM format:

[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-certutil ca --pem

After executing the command above, in the directory /../elasticsearch the archive will appear elastic-stack-ca.zip. Inside it you will find a certificate and a private key with extensions crt и key respectively. It is desirable to lay them out on a shared resource, to which there should be access from all nodes of the cluster.

Each node now needs its own certificates and private keys based on those in the shared directory. When the command is executed, you will be asked to set a password. Additional options -ip and -dns can be added to fully verify communicating nodes.

[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-certutil cert --ca-cert /shared_folder/ca/ca.crt --ca-key /shared_folder/ca/ca.key

As a result of the command execution, we will receive a certificate and a private key in the PKCS # 12 format, protected by a password. It remains to move the generated file p12 to the configuration directory:

[elastic@node1 ~]$ mv elasticsearch/elastic-certificates.p12 elasticsearch/config

Add a password to the certificate in the format p12 in keystore and truststore on each node:

[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

In the already known elasticsearch.yml it remains to add lines with data about the certificate:

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

We start all Elasticsearch nodes and execute curl. If everything was done correctly, a response with several nodes will be returned:

[elastic@node1 ~]$ curl node1:9200/_cat/nodes -u elastic:password                                                                                    
172.18.0.3 43 75 4 0.00 0.05 0.05 dim * node2                                                                                                                     
172.18.0.4 21 75 3 0.00 0.05 0.05 dim - node3                                                                                                                     
172.18.0.2 39 75 4 0.00 0.05 0.05 dim - node1

There is another security option - IP address filtering (available in subscriptions from the Gold level). Allows you to create white lists of IP addresses from which nodes are allowed to access.

Data security outside of an Elasticsearch cluster

Outside the cluster means connecting external instruments: Kibana, Logstash, Beats or other external clients.

To configure support for https (instead of http), add new lines to elasticsearch.yml:

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates.p12
xpack.security.http.ssl.truststore.path: elastic-certificates.p12

Because the certificate is password protected, let's add it to the keystore and truststore on each node:

[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password

After adding the keys, the Elasticsearch nodes are ready to connect via https. Now they can be launched.

The next step is to create a Kibana connection key and add it to the configuration. Based on the certificate, which is already located in the shared directory, we will generate a certificate in PEM format (PKCS # 12 Kibana, Logstash and Beats do not yet support):

[elastic@node1 ~]$ ./elasticsearch/bin/elasticsearch-certutil cert --ca-cert /shared_folder/ca/ca.crt --ca-key /shared_folder/ca/ca.key --pem

It remains to unpack the generated keys into the Kibana configuration folder:

[elastic@node1 ~]$ unzip elasticsearch/certificate-bundle.zip -d kibana/config

The keys are there, so it remains to change the Kibana configuration so that it starts using them. In the kibana.yml configuration file, change http to https and add lines with SSL connection settings. The last three lines set up a secure interaction between the user's browser and Kibana.

elasticsearch.hosts: ["https://${HOSTNAME}:9200"]
elasticsearch.ssl.certificateAuthorities: /shared_folder/ca/ca.crt
elasticsearch.ssl.verificationMode: certificate
server.ssl.enabled: true
server.ssl.key: /../kibana/config/instance/instance.key
server.ssl.certificate: /../kibana/config/instance/instance.crt

Thus, the settings are completed and access to data in the Elasticsearch cluster is encrypted.

If you have questions about the Elastic Stack features on free or paid subscriptions, tasks for monitoring or creating a SIEM system, leave a request in feedback form on our website.

More of our articles about Elastic Stack on Habré:

Understanding Machine Learning in the Elastic Stack (aka Elasticsearch, aka ELK)

Sizing Elasticsearch

Source: habr.com