Historically, the IT industry has been divided into two camps on any given topic: those who are 'for' and those who are 'against'. The subjects of debate can be completely arbitrary. Which OS is better: Windows or Linux? Should you use an Android smartphone or an iPhone? Is it better to store everything in the cloud or to upload to cold RAID storage and keep the drives in a safe? Do PHP developers have the right to call themselves programmers? These disputes can sometimes be purely existential in nature and have no basis other than sporting interest.
It just so happened that with the advent of containers and our beloved Docker and the somewhat conditional k8s, disputes 'for' and 'against' the use of new capabilities in various backend areas began. (Let’s state in advance that while Kubernetes is often mentioned as the orchestrator, the choice of this particular tool is not fundamentally important. You can substitute any other tool that you find most convenient and familiar.)
And it seemed this would be a simple debate between two sides of the same coin. Just as senseless and unforgiving as the eternal standoff of Windows vs Linux, where reasonable people exist somewhere in the middle. However, with containerization, it’s not that simple. Typically, there’s no right side in such debates, but in the case of 'to apply' or 'not to apply' containers for storing databases, everything turns upside down. Because, in a certain sense, both the supporters and opponents of this approach are right.
The Light Side
The argument of the Light Side can be summarized in one phrase: 'Hello, 2019 is outside!' It may sound like populism, but diving into the details reveals its benefits. Let's break those down now.
Suppose you have a large web project. It may have originally been built using a microservices approach, or it may have evolved into one at some point—this is not very important, really. You spread your project across individual microservices, set up orchestration, load balancing, and scaling. And now, with a clear conscience, you’re sipping mojitos in a hammock during HubSpot effects instead of raising fallen servers. But consistency is key in all actions. Very often, only the application itself—the code—is containerized. But what else do we have besides the code?
Correct, data. The heart of any project is its data: this can be a typical database like MySQL, PostgreSQL, MongoDB, or storage used for searching (ElasticSearch), key-value stores for caching—such as Redis, etc. We won't talk about poor backend implementation options where the database crashes due to badly written queries; instead, let’s discuss ensuring the fault tolerance of this database under client load. After all, when we containerize our application and allow it to scale freely to handle any number of incoming requests, this naturally increases the load on the database.
In fact, the channel of communication with the database and the server it runs on become the bottleneck in our beautifully containerized backend. The main motive of container virtualization is the mobility and flexibility of the structure, allowing for the organization of peak load distribution across all available infrastructure as efficiently as possible. In other words, if we do not containerize and deploy all existing system elements across a cluster, we are making a very serious mistake.
It makes much more sense to cluster not only the application itself but also the services responsible for data storage. When clustering and deploying independently operating and load-distributing web servers in k8s, we are already solving the data synchronization problem — such as comments to posts, for instance, when considering some media or blogging platform. In any case, we create an intra-cluster, albeit virtual, representation of the database as an ExternalService. The issue is that the database itself is not yet clustered — the deployed web servers in the cluster retrieve information about changes from our static production database, which runs separately.
Do you sense a catch? We use k8s or Swarm to distribute the load and avoid crashing the main web servers, but we do not do this for the database. But if the database crashes, then there is no point in our clustered infrastructure — what use are empty web pages returning a database access error?
That is precisely why we need to cluster not only web servers, as is typically done, but also the database infrastructure. Only in this way can we ensure a fully functioning system that works together yet remains independent of one another. Even if half of our backend crashes under load — the other half will survive, and the database synchronization within the cluster and the ability to infinitely scale and deploy new clusters will help quickly reach the required capacities — if only there were racks in the data center.
Moreover, the distributed model of databases in clusters allows us to place the database where it is needed; if we are talking about a global service, it is quite illogical to run a web cluster somewhere in the San Francisco area while sending packets for database access to Podmoskovye and back.
Containerizing the database also allows us to align all system components at the same level of abstraction. This, in turn, enables management of the system directly from the code, by developers, without actively involving admins. Developers thought that a separate DBMS for a new sub-project was needed — easy! They wrote a yaml file, loaded it into the cluster, and it was done.
Naturally, internal operations are simplified dramatically. Tell me, how many times have you winced when a new team member dove into the production database? The one that you actually have, and is currently running? Of course, we’re all adults here, and somewhere there’s a fresh backup, and even further back—behind the shelf with your grandmother's pickles and old skis—there’s another backup, possibly even on cold storage, because once your office caught fire. But still, every time a new team member gets access to the production infrastructure and, of course, the production database, it’s a bucket of Valium for everyone involved. Who knows this newcomer? What if they're clumsy? It’s scary, isn’t it?
Containerization and, essentially, the distributed physical topology of your project’s database helps avoid such moments of panic. Don't trust the newcomer? No problem! We'll set up their own cluster for work and disconnect it from the other database clusters—synchronization only through manual push and the synchronous turning of two keys (one for the team lead, the other for the admin). And everyone’s happy.
Now it’s time to switch to the arguments against database clustering.
The Dark Side
Let's discuss why you might not want to containerize your database and continue running it on a single central node. server, we won't stoop to the rhetoric of the orthodoxy and statements like 'our ancestors ran databases on hardware, and so shall we!' Instead, let’s try to think of a scenario where containerization would indeed bring tangible benefits.
Admit it, the projects that truly need a database in a container can be counted on the fingers of one not-so-great milling machine operator. For the most part, even the use of k8s or Docker Swarm can be excessive—often these tools are employed because of the overall hype around the technologies and the ‘supreme’ directives from the higher-ups to push everything into clouds and containers. Well, because it’s fashionable now and everyone is doing it.
In at least half of the cases, using Kubernetes or simply Docker for a project is excessive. The issue is that not all teams or outsourcing companies hired to maintain a client's infrastructure realize this. Worse still is when containers are imposed because this incurs a certain cost for the client.
There is a prevailing opinion that the Docker/K8s mafia simply monopolizes clients who outsource these infrastructure issues. After all, to work with clusters, engineers who are capable and understand the architecture of the implemented solution are needed. We have previously described our case with the publication Republic — there, we trained the client’s team to work in a Kubernetes environment, and everyone was satisfied. It was quite reasonable. However, often the 'implementers' of K8s hold the client's infrastructure hostage — because now only they understand how everything works, while there are no specialists on the client's side.
Now imagine that we are not only handing over the web server part to outsourcing, but also the management of the database. We said that the database is the heart, and losing a heart is fatal for any living organism. In short, the prospects are not the best. So instead of the hype around Kubernetes, many projects would be better off not being stingy on a decent AWS plan that will resolve all their load issues on their site/project. But AWS is no longer trendy, and showing off is more expensive than money — unfortunately, this is true in the IT environment as well.
Okay. Perhaps clustering is actually necessary for the project, but while the concept of stateless applications is clear, how then should one ensure adequate network connectivity for a clustered database?
When we talk about a seamless engineering solution, which the transition to k8s represents, our main headache is data replication in a clustered database. Some database management systems are initially quite permissive regarding data distribution among their individual instances. However, many others are not so accommodating. Quite often, the main argument in choosing a database management system for our project is not its ability to replicate with minimal resource and engineering costs, especially if the project was not initially planned as microservices but rather evolved in that direction.
We believe that there’s no need to elaborate on the speed of network disks—they are slow. In other words, we still have no real opportunity to relaunch a database instance somewhere with, for example, more CPU power or available RAM. We quickly hit the performance ceiling of the virtualized disk subsystem. Consequently, the database management system should be firmly tied to its own personal set of machines that are in close proximity. Alternatively, we need to find a way to potentially secure fast data synchronization with intended backups.
Continuing the topic of virtual file systems: Docker Volumes are unfortunately not without problems. In general, when it comes to long-term reliable data storage, we would like to operate with maximally straightforward technical schemes. Adding a new layer of abstraction from the container's file system to the host's parent file system is already a risk in itself. But when there are difficulties with data translation between these layers within the containerization support system, it becomes a serious issue. Currently, most of the known problems that humanity is aware of seem to have been eradicated. But as you understand, the more complex the mechanism, the easier it is for it to fail.
In light of all these "adventures," it is much more profitable and simpler to keep the database in one place. Even if you need containerization for the application, let it run independently and connect to the database through a distribution gateway, which will be read and written only once and in one place. This approach minimizes the likelihood of errors and desynchronizations.
What are we getting at? That database containerization is relevant where there is a real need for it. You can't just stuff a full-app database and run it as if you have two dozen microservices — that doesn't work. This must be clearly understood.
Instead of output
If you're expecting a clear answer on "to virtualize or not to virtualize the database," we're sorry: there won't be one. Because when creating any infrastructure solution, one must be guided not by trends and progress, but primarily by common sense.
There are projects where the principles and tools that come with Kubernetes fit perfectly, and in such projects, peace is achieved at least in the backend area. Then there are projects that need proper server infrastructure, as they fundamentally cannot scale to a microservices cluster model, or they will fail.
Source: habr.com
