🥇How to build a hybrid cloud with Kubernetes that can replace DBaaS

My name is Peter Zaitsev, I am the CEO and founder Percona and I want to share:

how we transitioned from open source solutions to Database as a Service;
what approaches exist for deploying databases in the cloud;
how Kubernetes can replace DBaaS, eliminating vendor dependence while maintaining the simplicity of a DBMS as a service.

This article is based on a presentation at the @Databases Meetup by Mail.ru Cloud Solutions & Tarantool. If you don't want to read, you can watch:

How We Transitioned from Open Source to Database as a Service in the Cloud

I have been involved with open source since the late 90s. Twenty years ago, using open source, such as databases, was not so straightforward. You had to download the source code, patch it, compile it, and only then use it.

Then open source went through a series of simplifications:

source code Tar.gz and INSTALL that needed to be compiled;
dependency packages like .deb and .rpm where you only needed to install a set of packages;
package repositories like APT and YUM, through which installation occurs automatically;
solutions like Docker and Snap that allow you to obtain packages for installation without external dependencies.

As a result, using open source software has become easier, and the barrier to entry for developing such applications has decreased.

While, unlike 20 years ago when everyone was an expert in building, most developers today cannot compile from source the tools they use.

Actually, this is not a bad thing because:

We can use more complex but more convenient software. For instance, a browser is convenient to use, but it includes many open source components, and it's inconvenient to build it from scratch.
More people can become developers of open source and other software, more software is utilized by businesses, leading to a higher demand for it.

The downside is that the next step in simplification is related to the use of cloud solutions, which brings about certain vendor lock-in, meaning attachment to a single provider. We use simple solutions, and providers use open source components, but they are effectively nailed to one of the major clouds. Therefore, the simplest and quickest way to deploy open source (and compatible software) is in the clouds, using proprietary APIs.

When it comes to databases in the cloud, there are two approaches:

Build a database infrastructure like that of a traditional data center. This means taking standard building blocks: compute, storage, and so on, installing Linux, the database, and configuring it.
Utilize Database as a Service, where the provider offers a ready-made database within the cloud.

DBaaS is currently a rapidly growing market because it allows developers to work directly with databases and minimizes routine tasks. The provider takes on the responsibility for High Availability and easy scalability, database patching, backups, and performance tuning.

Two types of Database as a Service based on open source and an alternative in the form of Kubernetes.

There are two types of Database as a Service for open databases:

A standard open source product packaged into a backend for administration, simplifying deployment and management.
An advanced commercial solution with various add-ons, compatible with open source.

Both options reduce the ability to migrate between clouds, decreasing the portability of data and applications. For example, although different types of clouds essentially support the same standard MySQL, there are significant differences: in operation, performance, backup, and so on. Migrating from one cloud to another can be challenging, especially for complex applications.

This raises the question—can we get the convenience of Database as a Service, but as a simple open source solution?

The bad news is that, unfortunately, there are currently no such solutions on the market. The good news is that Kubernetes exists, which allows such solutions to be implemented.

Kubernetes is an operating system for the cloud or data center that enables the deployment and management of applications across multiple cluster servers rather than on a single host.

Currently, Kubernetes is the leader in the category of such software. There have been many different solutions for these tasks, but it has become the standard. Many companies that previously worked on alternative solutions are now focusing on adapting their products to support Kubernetes.

In addition, Kubernetes is a universal solution supported in private, public, and hybrid clouds from many vendors, such as AWS, Google Cloud, Microsoft Azure. Mail.ru Cloud Solutions.

How Kubernetes works with databases

Kubernetes was originally developed for stateless applications that process data but do not store it, such as microservices or web applications. Databases are at the other end of the spectrum, meaning they are stateful applications. Kubernetes was not initially designed for such applications.

However, there are features that have been introduced in Kubernetes recently, allowing the use of databases and other stateful applications:

The StatefulSet concept is a complete set of primitives for handling pod stop events and implementing Graceful Shutdown (predictable application termination).
Persistent Volumes are data stores that are linked to pods, Kubernetes management objects.
Operator Framework refers to the ability to create components for managing databases and other stateful applications distributed across multiple nodes.

There are already large Database as a Service offerings in public clouds, powered by Kubernetes, such as CockroachCloud, InfluxDB, and PlanetScale. This means that running a database on Kubernetes is not just theoretically possible, but is also practically viable.

Percona offers two open-source solutions for Kubernetes:

Kubernetes Operator for Percona Server for MongoDB.
Kubernetes Operator for XtraDB CLUSTER — a service compatible with MySQL that provides high availability and consistency. It can also be used in single-node mode if high availability is not required, for example, for a dev database.

Kubernetes users can be divided into two groups. Some use Kubernetes Operators directly — these are mainly advanced users who understand how the technology works. Others run it in the backend — these users are interested in something like Database as a Service, and do not want to delve into the intricacies of Kubernetes. For the second group of users, we have another open-source solution — Percona DBaaS CLI Tool. This experimental solution is for those who want to obtain an open-source DBaaS based on Kubernetes without a deep understanding of the technology.

How to run Percona's DBaaS on Google Kubernetes Engine

Google Kubernetes Engine, in my opinion, is one of the most functional implementations of the Kubernetes technology. It is available in many regions around the world and features a simple and convenient Command Line Tool (SDK) that allows you to create scripts instead of managing the platform manually.

In order for our DBaaS to work, the following components are needed:

Kubectl.
Google Cloud SDK.
Percona DBaaS CLI.

Installing kubectl

Installing the package for your operating system; we'll use Ubuntu as an example. Learn more here.

sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

Installing Google Cloud SDK

Similarly, install the software package. Learn more here.

# Add the Cloud SDK distribution URI as a package source
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] 
http://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

# Import the Google Cloud Platform public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

# Update the package list and install the Cloud SDK
sudo apt-get update && sudo apt-get install google-cloud-sdk

Installing Percona DBaaS CLI

Install from Percona repositories. The Percona DBaaS CLI Tool is currently experimental, so it resides in an experimental repository that must be enabled separately, even if you already have Percona repositories installed.

Learn more here.

Installation steps:

Configure Percona repositories using the percona-release tool. First, you need to download and install the official percona-release package from Percona:
```
wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
```
Enable the experimental component of the tools repository as follows:
```
sudo percona-release enable tools experimental
```

Install the percona-dbaas-cli package:

sudo apt-get update
sudo apt-get install percona-dbaas-cli

Configuring component operation

More about settings here.

First, you need to log in to your Google account. Google Cloud allows a single user to have many independent projects, so you need to specify the active project using its code:

gcloud auth login
gcloud config set project hidden-brace-236921

Next, create a cluster. For the demo, I created a Kubernetes cluster with just three nodes — the minimum required for high availability:

gcloud container clusters create --zone us-central1-a your-cluster-name --cluster-version 1.15 --num-nodes=3

The following kubectl command grants the necessary privileges to our current user:

kubectl create clusterrolebinding cluster-admin-binding-$USER 
--clusterrole=cluster-admin --user=$(gcloud config get-value core/account)

Then we create a namespace and make it active. A namespace is basically similar to a project or environment, but within the Kubernetes cluster. It is independent of Google Cloud projects:

kubectl create namespace my-namespace
kubectl config set-context --current --namespace=my-namespace

Starting the cluster

After we've gone through these steps, we can start a three-node cluster with this simple command:

# percona-dbaas mysql create-db example
Starting ......................................... [done]
Database started successfully, connection details are below:
Provider:          k8s
Engine:            pxc
Resource Name:     example
Resource Endpoint: example-proxysql.my-namespace.pxc.svc.local
Port:              3306
User:              root
Pass:              Nt9YZquajW7nfVXTTrP
Status:            ready

How to connect to the cluster

By default, it is only accessible within Kubernetes. This means that from the server where you executed the 'Create' command, it is not available. To make it accessible, for example, for client testing, you need to expose the port via Port Mapping:

kubectl port-forward svc/example-proxysql 3306:3306 $

Then connect your MySQL client:

mysql -h 127.0.0.1 -P 3306 -uroot -pNt9YZquajW7nfVXTTrP

Advanced cluster management commands

Database on a public IP

If you're looking for a more permanent solution for cluster availability, you can obtain an external IP address. In this case, the database will be accessible from anywhere. It's less secure, but often more convenient. Use the following command for the external IP:

# percona-dbaas mysql create-db exposed 
--options="proxysql.serviceType=LoadBalancer"
Starting ......................................... [done]
Database started successfully, connection details are below:
Provider:          k8s
Engine:            pxc
Resource Name:     exposed
Resource Endpoint: 104.154.133.197
Port:              3306
User:              root
Pass:              k0QVxTr8EVfgyCLYse
Status:            ready

To access database please run the following command:
mysql -h 104.154.133.197 -P 3306 -uroot -pk0QVxTr8EVfgyCLYse

Explicitly set the password

Instead of having the system generate a random password, you can explicitly set the password:

# percona-dbaas mysql create-db withpw --password=mypassword
Starting ......................................... [done]
Database started successfully, connection details are below:
Provider:          k8s
Engine:            pxc
Resource Name:     withpw
Resource Endpoint: withpw-proxysql.my-namespace.pxc.svc.local
Port:              3306
User:              root
Pass:              mypassword
Status:            ready

I am showing the output of scripts in a readable format, but JSON format is also supported.

Disable high availability

The following command can be used to disable high availability to deploy a single node:

# percona-dbaas mysql create-db singlenode 
--options="proxysql.enabled=false, allowUnsafeConfigurations=true,pxc.size=1"
Starting ......................................... [done]
Database started successfully, connection details are below:
Provider:          k8s
Engine:            pxc
Resource Name:     singlenode
Resource Endpoint: singlenode-pxc.my-namespace.pxc.svc.local
Port:              3306
User:              root
Pass:              22VqFD96mvRnmPMGg
Status:            ready

This solution is for testing tasks, to quickly and easily set up MySQL, test it, and then dismantle it or use it for development.

The Percona DBaaS CLI tool helps to obtain a Kubernetes solution similar to DBaaS. We continue to work on its functionality and usability.

This presentation was first given at @Databases Meetup by Mail.ru Cloud Solutions&Tarantool. See video other presentations and subscribe for event announcements on Telegram Around Kubernetes at Mail.ru Group.