In anticipation of the start of a new enrollment for the course prepared a translation of a useful article for you.
Transparent Data Encryption (TDE) appeared in and MySQL for quite some time now. But have you ever thought about how it works under the hood and what impact TDE can have on your server? In this article series, we'll take a look at how TDE works internally. Let's start with key storage, as it is required for any encryption to work. Then we will take a closer look at how encryption works in Percona Server for MySQL/MySQL and what additional features are available in Percona Server for MySQL.
MySQL Keyring
Keyrings are plugins that allow the server to query, create and delete keys in a local file (keyring_file) or on a remote server (for example, in HashiCorp Vault). Keys are always cached locally to speed up retrieval.
Plugins can be divided into two categories:
- Local storage. For example, a local file (we call this a file-based keyring).
- remote storage. For example Vault Server (we call it a server-based keyring).
This separation is important because different storage types behave slightly differently not only when storing and retrieving keys, but also when they start up.
When using file storage, all the contents of the storage are loaded into the cache at startup: key id, key user, key type, and the key itself.
In the case of a back-end store (for example, the Vault server), only the key id and key user are loaded at startup, so getting all the keys does not slow down the startup. Keys are loaded lazily. That is, the key itself is loaded from Vault only when it is actually needed. After downloading, the key is cached in memory so that in the future there is no need to access it through TLS connections to the Vault Server. Next, consider what information is present in the keystore.
Key information contains the following:
- key id - key identifier, for example:
INNODBKey-764d382a-7324-11e9-ad8f-9cb6d0d5dc99-1 - key type - key type based on the encryption algorithm used, possible values: "AES", "RSA" or "DSA".
- key length - key length in bytes, AES: 16, 24 or 32, RSA 128, 256, 512 and DSA 128, 256 or 384.
- user is the owner of the key. If the key is a system key, such as Master Key, then this field is empty. If the key is created using keyring_udf, then this field indicates the owner of the key.
- the key itself
The key is uniquely identified by the pair: key_id, user.
There are also differences in key storage and deletion.
File storage is faster. You might think that the keystore is just a one-time write of the key to a file, but it's not - there's more going on here. Whenever a file storage is modified, all content is first backed up. Let's say the file is called my_biggest_secrets, then the backup will be my_biggest_secrets.backup. Next, the cache is changed (keys are added or removed) and, if everything is successful, the cache is dumped into a file. In rare cases, such as a server crash, you may see this backup file. The backup file is deleted the next time the keys are loaded (usually after a server restart).
When saving or deleting a key in the server store, the store must connect to the MySQL server with the "send the key" / "request key deletion" commands.
Let's get back to server startup speed. In addition to the fact that the vault itself affects the startup speed, there is also the question of how many keys from the vault need to be obtained at startup. Of course, this is especially important for server storages. On startup, the server checks which key is needed for the encrypted tables/tablespaces and requests the key from the storage. On a "clean" server with a Master Key - there must be one Master Key for encryption, which must be retrieved from the storage. However, more keys may be required, for example, when a backup from the primary server is restored to a standby server. In such cases, Master Key rotation should be provided. This will be discussed in more detail in future articles, although here I would like to note that a server using multiple Master Keys can take a little longer to start, especially when using a server key store.
Now let's talk a little more about keyring_file. When I was designing the keyring_file, I was also concerned about how to check for keyring_file changes while the server is running. In 5.7, the check was performed based on file statistics, which was not ideal, and in 8.0 it was replaced with a SHA256 checksum.
The first time keyring_file is run, the file statistics and checksum are calculated and remembered by the server, and changes are applied only if they match. When the file changes, the checksum is updated.
We have already covered many questions about keystores. However, there is another important topic that is often forgotten or misunderstood - sharing keys across servers.
What I mean? Each server (for example, Percona Server) in the cluster must have a separate location on the Vault Server in which Percona Server must store its keys. Each Master Key stored in the vault contains the Percona Server GUID within its ID. Why is it important? Imagine that you have only one Vault Server and all Percona Servers in the cluster use this single Vault Server. The problem seems obvious. If all Percona Servers used a Master Key with no unique identifiers, such as id = 1, id = 2, etc., then all servers in the cluster would use the same Master Key. This is what the GUID provides - the distinction between servers. Why then talk about sharing keys between servers if there is already a unique GUID? There is another plugin - keyring_udf. With this plugin, a user of your server can store their keys on the Vault server. The problem occurs when a user creates a key, for example, on server1, and then tries to create a key with the same ID on server2, for example:
--server1:
select keyring_key_store('ROB_1','AES',"123456789012345");
1
--1 Π·Π½Π°ΡΠΈΡ ΡΡΠΏΠ΅ΡΠ½ΠΎΠ΅ Π·Π°Π²Π΅ΡΡΠ΅Π½ΠΈΠ΅
--server2:
select keyring_key_store('ROB_1','AES',"543210987654321");
1Wait. Both servers use the same Vault Server, shouldn't the keyring_key_store function fail on server2? Interestingly, if you try to do the same on the same server, you will get an error:
--server1:
select keyring_key_store('ROB_1','AES',"123456789012345");
1
select keyring_key_store('ROB_1','AES',"543210987654321");
0That's right, ROB_1 already exists.
Let's discuss the second example first. As we said earlier, keyring_vault or any other vault plugin (keyring) caches all key IDs in memory. So, after creating a new key, ROB_1 is added to server1, and in addition to sending this key to Vault, the key is also added to the cache. Now, when we try to add the same key a second time, keyring_vault checks if the key exists in the cache and throws an error.
In the first case, the situation is different. Server1 and server2 have separate caches. After adding ROB_1 to the key cache on server1 and the Vault server, the key cache on server2 is out of sync. The cache on server2 does not contain the ROB_1 key. Thus, the ROB_1 key is written to the keyring_key_store and to the Vault server, which actually overwrites (!) the previous value. Now the ROB_1 key on the Vault server is 543210987654321. Interestingly, the Vault server does not block such actions and easily overwrites the old value.
Now we see why segregation by server in Vault can be important when you use keyring_udf and want to store keys in Vault. How to provide such separation in the Vault server?
There are two ways to partition on Vault. You can create different mount points for each server, or use different paths within the same mount point. The best way to show this is with examples. So let's look at the individual mount points first:
--server1:
vault_url = http://127.0.0.1:8200
secret_mount_point = server1_mount
token = (...)
vault_ca = (...)
--server2:
vault_url = http://127.0.0.1:8200
secret_mount_point = sever2_mount
token = (...)
vault_ca = (...)Here you can see that server1 and server2 use different mount points. With path separation, the configuration will look like this:
--server1:
vault_url = http://127.0.0.1:8200
secret_mount_point = mount_point/server1
token = (...)
vault_ca = (...)
--server2:
vault_url = http://127.0.0.1:8200
secret_mount_point = mount_point/sever2
token = (...)
vault_ca = (...)In this case, both servers use the same mount point "mount_point" but different paths. When you create the first secret on server1 using this path, the Vault server automatically creates a "server1" directory. For server2, everything is the same. When you delete the last secret in mount_point/server1 or mount_point/server2, the Vault server also deletes those directories. In case you are using split paths, you must create only one mount point and change the config files so that the servers use separate paths. A mount point can be created with an HTTP request. With CURL this can be done like this:
curl -L -H "X-Vault-Token: TOKEN" βcacert VAULT_CA
--data '{"type":"generic"}' --request POST VAULT_URL/v1/sys/mounts/SECRET_MOUNT_POINTAll fields (TOKEN, VAULT_CA, VAULT_URL, SECRET_MOUNT_POINT) correspond to the parameters of the configuration file. Of course, you can use the Vault utilities to do the same. But it's easier to automate the creation of a mount point. I hope you find this information helpful and see you in the next articles in this series.
Read more:
Source: habr.com