{"id":34450,"date":"2019-10-31T21:58:24","date_gmt":"2019-10-31T18:58:24","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kubernetes-zahvatit-mir-kogda-i-kak\/"},"modified":"2019-10-31T21:58:24","modified_gmt":"2019-10-31T18:58:24","slug":"kubernetes-zahvatit-mir-kogda-i-kak","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/kubernetes-zahvatit-mir-kogda-i-kak","title":{"rendered":"Kubernetes will take over the world. When and how?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

In anticipation of DevOpsConf<\/a><\/noindex> Vitaly Khabarov<\/b> interviewed\u00a0Dmitry Stolyarov<\/b> (distol<\/a><\/noindex>), the CTO and co-founder of the company \"Flant.\" Vitaly asked Dmitry about what \"Flant\" does, about Kubernetes, ecosystem development, and support. They discussed why Kubernetes is needed and if it is needed at all. Also, they talked about microservices, Amazon AWS, the \"I'll get lucky\" approach in DevOps, the future of Kubernetes itself, why, when, and how it will take over the world, the prospects of DevOps, and what engineers should prepare for in the near future with simplification and neural networks.<\/p>\n

The original interview<\/a><\/noindex> in podcast form can be listened to on DevOps DeFlop \u2014 a Russian-language podcast about DevOps, and below is the text version. <\/p>\n

\"Kubernetes
\n
\nHereafter, the questions are asked by Vitaly Khabarov<\/a><\/noindex> an engineer from Express42.
\n<\/a><\/noindex><\/p>\n

About \"Flant\"<\/h2>\n

\n\u2014 Dima, hello. You are the CTO of \"Flant<\/a><\/noindex>\" and also its founder. Please tell us what the company does and what your role is there?<\/b><\/p>\n

\"KubernetesDmitry<\/b>: From the outside, it seems like we are the kind of guys who go around setting up Kubernetes for everyone and doing something with it. But that's not true. We started as a company focused on Linux, but for a long time, our main activity has been servicing production and high-load projects turnkey. Usually, we build the entire infrastructure from scratch and then take full responsibility for it for a long time. Therefore, the main work that \"Flant\" performs, for which we get paid, is taking responsibility and implementing production turnkey.<\/b>.
\n

\n

\nAs the CTO and one of the founders of the company, I work around the clock to come up with ways to increase production availability, simplify its operation, ease the lives of admins, and make life easier and more pleasant for developers.<\/p>\n

About Kubernetes<\/h2>\n

\n\u2014 Lately, I've seen a lot of talks from \"Flant\" about Kubernetes. How did you come to it?\u00a0articles<\/a><\/noindex> : I've told this many times before, but I don't mind repeating it at all. I believe it's important to keep repeating this topic because confusion arises between cause and effect.<\/b><\/p>\n

Dmitry<\/b>: I have already talked about this many times, but I don't mind repeating it at all. I believe it is important to revisit this topic because there is confusion between cause and effect.<\/p>\n

We really needed a tool. We faced a lot of problems, struggled, overcame them with various workarounds, and felt the need for a proper tool. We explored many options, built our own solutions, and gained experience. Gradually, we began using Docker almost as soon as it appeared\u2014around 2013. At the time of its emergence, we already had extensive experience with containers; we had even written our own equivalent of Docker\u2014some workarounds in Python. With Docker's arrival, we were able to discard those workarounds and use a reliable, community-supported solution.<\/p>\n

The story with Kubernetes is similar. By the time it started gaining traction\u2014specifically version 1.2\u2014we already had a bunch of workarounds in both Shell and Chef that we were trying to use to orchestrate Docker. We seriously considered Rancher and various other solutions, but then Kubernetes came along, perfectly implemented the ideas we had, or even did it better. There was nothing to criticize.<\/p>\n

Yes, there are some unfinished aspects here, and there are many issues\u20141.2 is particularly troublesome, but... Kubernetes is like a building under construction: you look at the project and realize it will be great. If the building currently has a foundation and two floors, you understand it\u2019s better not to move in just yet, but with the software, there are no such problems\u2014you can already use it.<\/p>\n

We never had a moment of doubt about whether to use Kubernetes or not. We were waiting for it long before it was released and tried to create our own analogs.<\/p><\/blockquote>\n

<\/p>\n

About Kubernetes<\/h2>\n

\n\u2014 Are you directly involved in the development of Kubernetes?<\/b><\/p>\n

Dmitry<\/b>: Moderately. We are more involved in the development of the ecosystem. We submit a number of pull requests: to Prometheus, various operators, Helm\u2014into the ecosystem. Unfortunately, I am unable to keep track of everything we do and may make mistakes, but we have not submitted a single pull request to the core.<\/p>\n

\u2014 At the same time, do you develop many of your own tools around Kubernetes?<\/b><\/p>\n

Dmitry<\/b>: The strategy is like this: we contribute and make pull requests to everything that already exists. If pull requests are not accepted, we simply fork those projects for ourselves and continue using them until they are accepted with our builds. Then, when that reaches upstream, we transition back to the upstream version.<\/p>\n

For example, we have the Prometheus operator, with which we switched back and forth to the upstream of our build about 5 times, I suppose. We need some feature, we sent a pull request, we need to roll it out tomorrow, and we don\u2019t want to wait for it to be released in upstream. Accordingly, we collect it for ourselves, rolling out our build with the feature that we need across all our clusters. Then, for instance, it gets brought back to upstream with the words: 'Guys, let\u2019s make this for a more general case,' and we, or someone else, finalizes it, and over time it flows back.<\/p>\n

We strive to develop everything that exists.<\/b>Many elements that either do not yet exist, have not been conceived, or have been conceived but not implemented \u2013 we create. And not because we like the process itself or building bicycles as an industry, but simply because we need this tool. We are often asked why we made this or that thing. The answer is simple \u2013 because we needed to move forward, solve some practical problem, and we resolved it with this tool.<\/p>\n

The path is always like this: we search very thoroughly and if we can\u2019t find any solution, like how to make a trolleybus out of a loaf of bread, then we make our own loaf and our own trolleybus.<\/p><\/blockquote>\n

<\/p>\n

Flant Tools<\/h2>\n

\nI know that Flant currently has addon operators, shell operators, and dapp\/werf tools. As I understand it, these are the same tool in different incarnations. I also understand that there are many different tools within Flant. Is that correct?<\/b><\/p>\n

Dmitry<\/b>We have a lot more on GitHub. From what I can recall right now, we have statusmap \u2013 a panel for Grafana that has been widely accepted. It is mentioned in almost every second article about monitoring Kubernetes on Medium. It's impossible to briefly describe what statusmap is \u2013 it requires a separate article, but it is a very useful thing for monitoring status over time, as in Kubernetes we often need to show the status over time. We also have LogHouse \u2013 a tool based on ClickHouse and some black magic for collecting logs in Kubernetes.<\/p>\n

Many utilities! And there will be even more, as some internal solutions will be released this year. Among the very large ones based on the addon operator, there are a bunch of addons for Kubernetes, such as how to properly set up cert-manager \u2014 a tool for managing certificates, or how to correctly set up Prometheus with a lot of extensions \u2014 a good twenty different binaries that export data and collect something, plus Prometheus offers amazing graphics and alerts. All of this is just a ton of addons for Kubernetes that are installed in the cluster, turning it from simple to sophisticated, feature-rich, and automatic, where many questions have already been resolved. Yes, we are doing a lot.<\/p>\n

Ecosystem Development<\/h2>\n

\n\u2014 I believe this is a significant contribution to the development of this tool and its methods of use. Can you estimate who else might contribute similarly to the ecosystem's development?<\/b><\/p>\n

Dmitry<\/b>: In Russia, among the companies operating in our market \u2014 no one is even close.<\/b>. Of course, this is a bold statement, because there are major players like Mail and Yandex \u2014 they are also doing something with Kubernetes, but even they haven't come close to the contributions made by companies globally, who are doing much more than we are. It's hard to compare 'Flant' with a team of 80 people and Red Hat, which has 300 engineers just for Kubernetes, if I'm not mistaken. It's tough to compare. We have 6 people in our RnD department, including me, who build all our tools. 6 people versus 300 engineers from Red Hat \u2014 it's a bit difficult to compare.<\/p>\n

\u2014 Nevertheless, when even these 6 people can create something genuinely useful and transferable, when they face a practical challenge and provide a solution to the community \u2014 it's an interesting case. I understand that in large tech companies, where they have their own development and support team for Kubernetes, similar tools can indeed be developed. This is an example for them that something can be developed and given to the community, giving a boost to the entire community that uses Kubernetes.<\/b><\/p>\n

Dmitry<\/b>This is probably the integrator's trick, its characteristic. We have many projects, and we see a variety of situations. Our main way to create added value is to analyze these cases, find commonalities, and maximize cost efficiency for us. We are actively engaged in this. It's hard for me to speak about Russia and the world, but we have about 40 DevOps engineers in the company who work with Kubernetes. I don't think there are many companies in Russia with a comparable number of specialists knowledgeable about Kubernetes, if any exist at all.<\/p>\n

I understand everything about the title of DevOps Engineer; everyone understands it and has gotten used to calling DevOps engineers as such, we aren't going to discuss that. All 40 of these wonderful DevOps engineers face problems every day and solve them; we simply analyze this experience and try to generalize it. We understand that if this knowledge stays internal, then in a year or two the tool becomes useless because somewhere in the community a ready-made solution will appear. There's no sense in hoarding this experience internally\u2014it's just draining energy and time into dev\/null. But we absolutely don't mind. We publish everything with great pleasure and know that it needs to be published, developed, promoted, and circulated so that people can use it and add their own experiences\u2014then everything grows and thrives. In two years, the tool won't end up in the trash. We don't mind continuing to invest effort because it's clear that someone is using our tool, and in two years everyone will be using it.<\/p>\n

This is part of our larger strategy with dapp\/werf.<\/b>I don\u2019t remember when we started working on it, it seems like three years ago. Initially, it was done entirely in shell. It was a super proof of concept, addressing some of our specific tasks\u2014it worked! But there are problems with shell; it's impossible to scale further, and programming in shell is quite the ordeal. We had a habit of writing in Ruby, so we transformed some aspects and kept developing it until we encountered the issue that the community, the crowd that doesn\u2019t say \u2018we want or don\u2019t want,\u2019 turns its nose up at Ruby, however funny that may seem. We realized we needed to write all this in Go, just to meet the first item on the checklist: The DevOps tool must be a static binary.<\/b>Whether it's Go or not is not that important, but it's better to have a static binary written in Go.<\/p>\n

We invested effort, rewrote the dapp in Go, and named it werf. The dapp is no longer supported, not being developed, and works on some last version, but there is an absolute upgrade path upward that one can follow.<\/p>\n

Why was the dapp created<\/h2>\n

\n\u2014 Can you briefly explain why the dapp was created and what problems it solves?<\/b><\/p>\n

Dmitry<\/b>: The first reason is the build. Initially, we had significant problems with building when Docker wasn\u2019t capable of multi-stage builds, and we had to implement multi-stage on our own. Then, we had a ton more questions with cleaning up images. Everyone dealing with CI\/CD sooner or later faces the issue of having a lot of built images, and we need to clean up what\u2019s unnecessary and keep what is needed.<\/p>\n

The second reason is deployment. Yes, there's Helm, but it only addresses part of the tasks. Strangely enough, it\u2019s stated that 'Helm is the Package Manager for Kubernetes.' Specifically, 'the.' There are also the words 'Package Manager' \u2014 what is usually expected from a Package Manager? We say, 'Package Manager \u2013 install the package!' and expect it to tell us: 'The package has been installed.' <\/p>\n

Interestingly, we say, 'Helm, install the package,' but when it responds that it installed it, it turns out that it only started the installation \u2014 it told Kubernetes: 'Run this thing!' But whether it started or not, whether it works or not, Helm does not address this question at all.<\/p>\n

It results in Helm being just a text preprocessor that uploads data to Kubernetes.<\/p><\/blockquote>\n

\nHowever, within any deployment, we want to know \u2014 did the application go live or not? Going live means that the application has been deployed there, a new version has rolled out, and at least it does not crash and responds correctly. Helm does not tackle this task at all. To solve it, a lot of effort is needed because it\u2019s necessary to give Kubernetes the command to roll it out and monitor what happens \u2014 whether it has rolled out or not. Plus, there are many tasks related to deployment, cleaning, and building.<\/p>\n

Plans<\/h2>\n

\nLater this year, we will move towards local development. We want to achieve what used to be possible with Vagrant \u2014 just typing 'vagrant up' would spin up virtual machines. We aim for a state where a project exists in Git, and we simply write 'werf up', which launches a local copy of that project, deployed in a local mini-Kub, with all directories connected for convenient development. Depending on the programming language, this is executed differently, but the goal is to facilitate smooth local development with mounted files.<\/p>\n

The next step for us is to heavily invest in developer convenience<\/b>. To quickly launch a project locally with one tool, develop it, push it to Git, and have it deployed to staging or testing based on the pipelines, and then use the same tool to go to production. This unity, unification, and reproducibility of the infrastructure from the local environment to production is a very important aspect for us. But this is not yet available in werf \u2014 it is something we are planning to implement.<\/p>\n

However, the path to dapp\/werf has always been similar to how we started with Kubernetes. We faced challenges and solved them with workarounds \u2014 coming up with various shell solutions, or anything else. Then we tried to streamline those workarounds, generalize them, and consolidate them into binaries that we simply share.<\/p>\n

There is also another perspective on this whole story, with analogies. <\/p>\n

Kubernetes is like a car frame with an engine. There are no doors, windows, radio, or anything \u2014 just the frame and the engine. And there\u2019s Helm \u2014 that\u2019s the steering wheel. It's great to have the steering wheel, but you also need the steering column, the steering rack, transmission, and wheels; without them, it's not functional.<\/p><\/blockquote>\n

\nIn the case of werf, it's another component for Kubernetes. Right now, we have an alpha version of werf, where, for example, Helm is actually compiled into werf, because we got tired of managing it separately ourselves. There are many reasons for doing it this way, and I will explain in detail why we integrated Helm completely along with Tiller into werf during my presentation at RIT++<\/a><\/noindex>.<\/p>\n

Now werf is a more integrated component. We get a ready steering wheel, a steering pin - I'm not very knowledgeable about cars, but it's a large module that addresses a significant range of tasks. We don't need to rummage through the catalog, picking one part to match another, figuring out how to fasten them together. We receive a ready-made combine that solves a large number of tasks right away. Yet inside, it is all built from the same open-source components, still using Docker for assembly, Helm for part of the functionality, and there are a few other libraries as well. It's an integrated tool for quickly and conveniently achieving great CI\/CD out of the box.<\/p>\n

Is it difficult to maintain Kubernetes?<\/h2>\n

\n\u2014 You're talking about the experience of starting to use Kubernetes; for you, it's a frame, an engine, and there are many different things you can attach to it: a body, a steering wheel, pedals, seats. The question arises - how challenging is it for you to maintain Kubernetes? You have a wealth of experience; how much time and resources do you spend specifically on maintaining Kubernetes, apart from everything else?<\/b><\/p>\n

Dmitry<\/b>: This is a very complex question, and to answer it, we need to understand what maintenance is and what we want from Kubernetes. Perhaps you can elaborate?<\/p>\n

\u2014 As far as I know and as I see, many teams want to try Kubernetes these days. Everyone is jumping in and installing it on their own. I have the feeling that people don't always grasp the complexity of this system.<\/b><\/p>\n

Dmitry<\/b>: That's true.<\/p>\n

\u2014 How difficult is it to take Kubernetes and set it up from scratch so that it is production-ready?<\/b><\/p>\n

Dmitry<\/b>: How do you think it is to transplant a heart? I understand that's a compromising question. Using a scalpel without making mistakes isn't that difficult. If you're told where to cut and where to stitch, then the procedure itself isn't hard. The challenge is guaranteeing success each time.<\/p>\n

Setting up Kubernetes and making it work is easy: snap! \u2014 installed, there are plenty of installation methods. But what happens when problems arise?<\/p><\/blockquote>\n

\nQuestions always arise - what have we overlooked? What have we not done? Which Linux kernel parameters did we set incorrectly? My God, did we even specify them at all?! Which Kubernetes components did we install, and which did we not? Thousands of questions come up, and to answer them, you need to have spent 15-20 years immersed in this industry.<\/p>\n

I have a fresh example on this topic that can highlight the essence of the question \"Is it difficult to maintain Kubernetes?\" Some time ago, we seriously considered whether we should implement Cilium as the networking layer in Kubernetes.<\/p>\n

Let me explain what Cilium is. In Kubernetes, there are many different implementations of the networking subsystem, and one of them is really cool \u2014 it's Cilium. What's its essence? Some time ago, the ability to write hooks for the kernel was added, which intrude into the networking subsystem and various other subsystems, allowing to bypass large parts of the kernel.<\/p>\n

Historically, the Linux kernel has ip rout, net-filter, bridges, and many different old components that are 15, 20, or even 30 years old. Overall, they work well; everything is great, but now we've piled on containers, and it looks like a tower made of 15 bricks stacked on top of each other, while you're standing on it on one leg \u2014 it feels strange. This system has historically developed with many nuances, like an appendix in the body. In some situations, there are performance problems, for example.<\/p>\n

There is a wonderful BPF and the ability to write hooks for the kernel \u2014 the guys wrote their own hooks for the kernel. A packet comes into the Linux kernel; they take it out right at the entrance, process it as needed without bridges, without TCP, without the IP stack \u2014 in short, bypassing everything that's written in the Linux kernel, and immediately spit it out into the container.<\/p>\n

What did we get? Very cool performance, great features \u2014 it's just fantastic! But we look at this and see that on each machine there is a program that connects to the Kubernetes API and based on the data it receives from this API, generates C code and compiles binaries that are loaded into the kernel, so that these hooks work in kernel space.<\/p>\n

What will happen if something goes wrong? We don't know. To understand this, you need to read all this code, grasp all the logic, and that\u2019s incredibly complex. But, on the other hand, there are these bridges, net-filters, ip rout \u2014 I haven't read their sources, nor have the 40 engineers who work in our company. Maybe only a few understand certain parts.<\/p>\n

What\u2019s the difference? There\u2019s an IP route, a Linux kernel, and a new tool \u2014 what difference does it make? We don\u2019t understand either one. But we\u2019re afraid to use the new \u2014 why? Because if a tool has been around for 30 years, all the bugs have been found, all the pitfalls encountered; you don\u2019t need to know everything \u2014 it works like a black box, and it always works. Everyone knows where to insert their diagnostic screwdriver and when to run tcpdump. Everyone is well acquainted with diagnostic utilities and understands how this set of components operates within the Linux kernel \u2014 not how it's structured, but how to use it.<\/p>\n

But the incredibly cool Cilium is not 30 years old; it\u2019s still new. Kubernetes has the same issue, it\u2019s a copy. Both Cilium and Kubernetes can be installed seamlessly, but when something goes wrong in production, can you quickly figure out what went wrong in a critical situation? <\/p>\n

When we talk about how difficult it is to maintain Kubernetes \u2014 it\u2019s not, it\u2019s very simple, and yes, it\u2019s incredibly complicated. Kubernetes works wonderfully by itself, but there are billions of nuances.<\/p><\/blockquote>\n

<\/p>\n

About the approach \"I\u2019ll get lucky\"<\/h2>\n

\n\u2014 Are there companies where these nuances will almost certainly appear? Let\u2019s suppose Yandex suddenly migrates all its services to Kubernetes; there will be quite a load there.<\/b><\/p>\n

Dmitry<\/b>: No, this conversation isn\u2019t about the load; it\u2019s about simple things. For instance, we have Kubernetes, and we\u2019ve deployed an application there. How do we know it\u2019s working? There\u2019s no ready-made tool to confirm that the application isn\u2019t crashing. There\u2019s no ready system that sends alerts \u2014 you have to set up those alerts and every graph. And we are updating Kubernetes.<\/p>\n

Ubuntu 16.04 is available. It can be considered an old version, but we are still using it because it is LTS. It has systemd, which has the nuance that it does not clean up C groups. Kubernetes launches pods, creates C groups, then deletes the pods, and somehow it turns out\u2014I don't remember the details, sorry\u2014that systemd slices remain. This leads to the fact that over time, any machine begins to lag significantly. This is not even a matter of high load. If persistent pods are launched, for example, if there is a Cron Job that continuously generates pods, then a machine with Ubuntu 16.04 will start to lag after a week. There will constantly be a high load average due to the creation of a lot of C groups. This is a problem faced by anyone who simply installs Ubuntu 16 and runs Kubernetes on top of it.<\/p>\n

Let's say he somehow updates systemd or something else, but in the Linux kernel before 4.16, it's even funnier\u2014when C groups are deleted, they leak in the kernel and actually are not removed. Therefore, after a month of work on this machine, it will be impossible to view memory statistics by pods. We pull out a file, process it, and one file takes 15 seconds to process because the kernel takes a long time to count internally through a million C groups that seem to be deleted, but they are leaking.<\/p>\n

There are still a lot of such minor issues here and there. This is not a question that giant companies might occasionally encounter under very high loads\u2014no, this is a matter of everyday occurrences. People can live like this for months\u2014they installed Kubernetes, deployed the application\u2014it seems to work. For many, this is normal. They won't even know that at some point this application will crash, the alert won't come, but for them, this is the norm. They used to live on virtual machines without monitoring, now they've moved to Kubernetes also without monitoring\u2014what's the difference?<\/p>\n

The question is that when we walk on ice, we never know its thickness unless we measured it beforehand. Many walk and don't worry because they walked on it before.<\/p>\n

From my point of view, the nuance and complexity of operating any system is to ensure that the thickness of the ice is certainly enough to meet our tasks. That's what it's about.<\/p><\/blockquote>\n

\nIn IT, it seems to me that there are too many approaches of \u2018I will get lucky.\u2019 Many set up software, use libraries in the hope that they will get lucky. Overall, many do get lucky. Probably that's why it works.<\/p>\n

\u2014 From my pessimistic assessment, it looks like this: when risks are high, and the application must work, support from \u2018Flant\u2019 is needed, possibly from Red Hat, or an internal team dedicated specifically to Kubernetes is required, ready to handle it.<\/b><\/p>\n

Dmitry<\/b>: Objectively, that's the case. Diving into the Kubernetes story alone for a small team involves a certain amount of risks.<\/p>\n

Do we need containers?<\/h2>\n

\n\u2014 Can you tell me how widespread Kubernetes is in Russia?<\/b><\/p>\n

Dmitry<\/b>: I don\u2019t have that data, and I\u2019m not sure if anyone does. We keep saying: \u2018Kubernetes, Kubernetes,\u2019 but there\u2019s another perspective on this issue. I also don\u2019t know how widespread containers are, but I know from reports online that 70% of containers are orchestrated by Kubernetes. That was from a reliable source based on a fairly large sample globally.<\/p>\n

Another question is\u2014do we need containers?<\/b> My personal feeling and the overall stance of \u2018Flant\u2019 is that Kubernetes is the de facto standard.<\/p>\n

Nothing but Kubernetes will exist.<\/p><\/blockquote>\n

\nThis is an absolute game-changer in the field of infrastructure management. Just absolutely\u2014no more Ansible, Chef, virtual machines, Terraform. I\u2019m not even talking about the old, outdated methods. Kubernetes is an absolute changer<\/b>, and from now on, it will only be this way.<\/p>\n

It is clear that some will need a couple of years, while others may take several decades to realize this. I have no doubt that there will be nothing but Kubernetes and this new perspective: we no longer run the operating system, but use infrastructure as code<\/b>, except not with code, but with yml\u2014declaratively described infrastructure. I have the feeling that it will always be this way.<\/p>\n

\u2014 So those companies that haven\u2019t yet transitioned to Kubernetes will inevitably do so or be forgotten. Did I understand you correctly?<\/b><\/p>\n

Dmitry<\/b>: This is not entirely correct either. For instance, if we need to set up a DNS server, we can run it on FreeBSD 4.10, and it can work perfectly fine for 20 years. Just work, that's it. Perhaps, over 20 years, something might need to be updated just once. If we talk about software in the format where we launched it and it actually runs for many years without any updates or changes, then, of course, Kubernetes won\u2019t be there. It\u2019s simply not needed.<\/p>\n

Anything related to CI\/CD \u2014 wherever Continuous Delivery is required, where version updates and active changes are needed, wherever resilience must be built \u2014 it\u2019s only Kubernetes.<\/p><\/blockquote>\n

<\/p>\n

About microservices<\/h2>\n

\n\u2014 Here I have a small dissonance. To work with Kubernetes, internal or external support is required \u2014 that\u2019s the first point. The second \u2014 when we are just starting development, we are a small startup, we still have nothing, developing for Kubernetes or even for microservices architecture can be complex, and not always economically justified. I\u2019m curious about your opinion \u2014 should startups from scratch immediately start developing for Kubernetes or can they write a monolith first and then move to Kubernetes?<\/b><\/p>\n

Dmitry<\/b>: Great question. I have a talk about microservices \"Microservices: size matters.\"<\/a><\/noindex> I\u2019ve encountered many times that people are trying to hammer nails with a microscope. The approach itself is correct, we design our internal software exactly in this way. But when doing this, you need to clearly understand what you\u2019re doing. What I hate the most about microservices is the word \"micro.\" Historically, this word emerged, and for some reason, people think that micro means very small, less than a millimeter, like a micrometer. That is not the case.<\/p>\n

For example, there\u2019s a monolith written by 300 people, and everyone involved in the development understands there are problems, and it needs to be broken down into smaller pieces \u2014 about 10, each written by at least 30 people. This is important, necessary, and great. But when a startup comes to us, where 3 very cool and talented guys have crudely written 60 microservices, each time I'm looking for a sedative.<\/p>\n

It seems to me that this has been discussed thousands of times \u2014 a distributed monolith has emerged in one form or another. This is economically unjustifiable and very complicated overall. I've seen this so many times that it hurts, which is why I continue to talk about it.<\/p>\n

To the initial question, there is a conflict between the fact that, on one hand, Kubernetes is frightening to use because it\u2019s unclear what might break or not work, and on the other hand, it\u2019s clear that everything is moving in that direction and nothing but Kubernetes will remain. The answer is \u2014 to weigh the amount of benefit that comes against the amount of tasks you can solve.<\/b>This is on one side of the scale. On the other side are the risks associated with downtime or reduced response times, levels of availability \u2014 with decreased performance metrics.<\/p>\n

It\u2019s either that we move quickly, and Kubernetes allows many things to be done much faster and better, or we use reliable, time-tested solutions but move much slower. This choice each company must make. You can think of it as a path in the jungle \u2014 the first time you walk it, you might encounter a snake, a tiger, or a rabid badger, but after walking it ten times, you pave the path, clear the branches, and it becomes easier to walk. With each time, the path gets wider. Eventually, it becomes a paved road, and later a beautiful boulevard.<\/p>\n

Kubernetes does not stand still. Again the question: Kubernetes, on one hand, consists of 4-5 binaries, and on the other hand, it\u2019s the entire ecosystem. It\u2019s the operating system that runs on our machines. What is it? Ubuntu or Curios? It\u2019s the Linux kernel, a bunch of additional components. All these things here \u2014 they removed a poisonous snake from the road, there they put up a fence. Kubernetes is rapidly and dynamically evolving, and the volume of risks, the amount of the unknown decreases each month, and accordingly, this scale gets rebalanced.<\/p>\n

In response to the question of what a startup should do, I would say \u2014 come to 'Flant', pay 150,000 rubles, and get a turnkey DevOps easy service. If you are a small startup with a few developers, this works. Instead of hiring your own DevOps who will need to learn how to solve your problems while you pay them a salary, you will get a turnkey solution for all issues. Yes, there are some downsides. As an outsourcer, we cannot be as involved and respond quickly to changes. But we have tons of expertise and ready practices. We guarantee that in any situation we will manage to quickly figure things out and revive any Kubernetes from the dead. <\/p>\n

I categorically recommend outsourcing for startups and established businesses until you can allocate a team of 10 people for operations, because otherwise it makes no sense. It absolutely makes sense to outsource.<\/p><\/blockquote>\n

<\/p>\n

About Amazon and Google<\/h2>\n

\n\u2014 Can hosting solutions from Amazon or Google be considered as outsourcing?<\/b><\/p>\n

Dmitry<\/b>: Yes, of course, this addresses a number of issues. But again, there are nuances. You still need to understand how to use it. For example, there are a thousand little details in working with Amazon AWS: the Load Balancer needs to be warmed up or you have to submit a request in advance saying, 'Guys, we will get traffic, please warm up our Load Balancer!' You need to know these nuances.<\/p>\n

When you consult people who specialize in this, you get nearly all standard issues addressed. We currently have 40 engineers, and by the end of the year there will likely be 60 \u2014 we have definitely encountered all these things. Even if we face this problem on some project again, we quickly ask each other and know how to solve it.<\/p>\n

Perhaps the answer is that, of course, hosted solutions alleviate some part. The question is whether you are ready to trust these hosts, and will they solve your problems. Amazon and Google have established a good reputation. For all our cases \u2014 definitely. We have no additional positive experiences. All other clouds we tried to work with create a lot of problems \u2014 whether it\u2019s Ager, all options available in Russia, or various OpenStack implementations: Headster, Overage \u2014 whatever you want. They all create problems that are undesirable to solve.<\/p>\n

Therefore, the answer is yes, but, in fact, there aren't many mature hosted solutions.<\/p>\n

Who needs Kubernetes?<\/h2>\n

\n\u2014 But seriously, who needs Kubernetes? Who should switch to Kubernetes, and who is a typical client of \"Flant\" that comes specifically for Kubernetes?<\/b><\/p>\n

Dmitry<\/b>: This is an interesting question because right now, many are coming to us in the wave of Kubernetes: \"Guys, we know you do Kubernetes, make it for us!\" We reply: \"Gentlemen, we don't do Kubernetes; we do production and everything related to it.\" Because creating production without setting up the entire CI\/CD and all that entails is simply impossible these days. Everyone has moved away from the separation of development being development and operations being operations.<\/p>\n

Our clients expect various things, but they all hope for some kind of miracle, that they have certain problems, and now\u2014voila!\u2014Kubernetes will solve them. People believe in miracles. While they understand rationally that there won't be a miracle, they still hope\u2014what if this Kubernetes solves everything for us; there's so much talk about it! What if it\u2019s now, poof!\u2014the silver bullet, poof!\u2014and we have 100% uptime, all developers can release something to production 50 times, and it won't crash. In general, a miracle!<\/p>\n

When such people come to us, we say: \"Sorry, but miracles don\u2019t happen.\" To be healthy, you need a balanced diet and to exercise. To have a reliable production environment, it needs to be built reliably. To have a convenient CI\/CD, it needs to be set up properly. This is a lot of work that needs to be done.<\/p>\n

To answer the question of who needs Kubernetes\u2014Kubernetes is not needed by anyone.<\/p><\/blockquote>\n

\nSome people have a mistaken impression that they need Kubernetes. What people really need is a deep desire to stop thinking, worrying, or being concerned about all the infrastructure problems and the issues of launching their applications. They want their applications to just work and be deployed easily. For them, Kubernetes is the hope that they will stop hearing about stories like \"we were stalled\" or \"we can't roll it out,\" or something else.<\/p>\n

The typical person who comes to us is a tech director. They are asked two things: on one hand, they want features; on the other hand, they want stability. We propose taking that burden on ourselves and handling it. The silver bullet\u2014more precisely, the silver-plated one\u2014is that you will stop thinking about these problems and wasting time. You will have dedicated people who will handle this issue.<\/p>\n

The statement that we or someone needs Kubernetes is incorrect.<\/p><\/blockquote>\n

\nKubernetes is very much needed by admins because it is a really interesting tool to play around with. Let's be honest \u2014 everyone loves toys. We are all like kids at some point, and when we see something new, we want to play with it. Some may have lost that interest, for example in administration, because they have already played enough and are simply tired of it. However, no one has completely lost that interest. For instance, while I may be tired of toys in system administration and DevOps, I still love toys and continue to buy new ones. People, in one way or another, always want some toys.<\/p>\n

Do not play with production. What I would categorically advise against, which I currently see happening widely: \"Oh, a new toy!\" \u2014 rush to buy it, then say: \"Let's introduce it here, so we can show it to all our friends.\" Don't do that. I apologize; I have kids growing up and I constantly notice something in them, see it in myself, and then generalize it to others.<\/p>\n

The final answer: you do not need Kubernetes. You need to solve your problems.<\/p><\/blockquote>\n

\nYou can achieve the following:<\/p>\n