🥇Is Docker just a toy or not? Or is it indeed?

Hello everyone!

I really want to dive straight into the topic, but it's better to share a bit about my story first:

Introduction

I am a programmer with experience in developing frontend single-page applications, and backend using Scala/Java and Node.js.

For quite some time (definitely a couple to three years), I believed that Docker is a godsend and an incredibly cool tool that every developer should know how to use. This leads to the conclusion that every developer should have Docker installed on their local machine. Just look at the job postings on sites like hh.ru. In every second one, there's mention of Docker, and if you know how to use it, that will be your competitive advantage 😉

Along the way, I encountered many people with varying attitudes towards Docker and its ecosystem. Some said it was a convenient tool that guarantees cross-platform compatibility. Others didn’t understand why they should run in containers and what the benefits were. Some didn’t care at all and just wrote code and went home – I actually envy them 🙂

Reasons for using it

Why did I use Docker? Probably for the following reasons:

to run databases, as 99% of applications use them
to run Nginx for serving the frontend and proxying to the backend
you can package the application into a Docker image, so my application will work anywhere Docker is present, solving the distribution problem right away
service discovery out of the box, you can create microservices where each container (connected to a common network) can easily reach another by alias, which is very convenient
it's fun to create a container and "play around" in it.

What I have always DISLIKED about Docker:

to run my application, Docker needs to be installed on the server. Why do I need that when my applications run on JRE or Node.js and the environment for them is already set up on the server?
if I want to run my (private) locally built image on a remote server, I need my own Docker repository, I need to have a registry running somewhere, and I also need to set up HTTPS because Docker CLI only works over HTTPS. Oh man… there are options, of course, to save the image locally and then simply transfer it via SCP… But it involves so much hassle. Plus, it seems like a "workaround" solution until you have your own repository. docker save and just use scp to drop the image... But that's so much hassle. Moreover, it looks like a "hacky" solution until you have your own repository.
docker-composeIt is only needed to run containers. That's all. It can't do anything else. Docker-compose It has a bunch of versions of its files, its own syntax. No matter how declarative it may be, I don't want to read their documentation. I won't need it anywhere else.
When working in a team, most people write Dockerfiles very poorly, not understanding how caching works, adding everything necessary and unnecessary to the image, inheriting from images that are not available on dockerhub or in private repositories, creating some docker-compose files with databases and not persisting anything. Meanwhile, developers proudly state that Docker is cool, everything works locally for them, and HR importantly writes in job postings: 'We use Docker, and we need a candidate with that experience.'
Constantly plagued by thoughts of bringing everything and anything up in Docker: PostgreSQL, Kafka, Redis. It's a shame not everything works in containers, not all is easy to configure and run. This is supported by third-party developers rather than the vendors themselves. And by the way, the question arises immediately: the vendors don't care about maintaining their products in Docker, why is that? Maybe they know something?
There is always a question about the persistence of container data. And here you think, should I just mount a host directory or create a Docker volume or make a data container which now deprecated? Если я монтирую директорию то мне нужно убедиться что uid и gid пользователя в контейнере соответствует id пользователя запустившего контейнер, иначе файлы созданные контейнером будут созданы с правами владельца root. Если использую volume will simply create data in some random /usr/* and there will be the same story with UID and GID as in the first case. If you run a third-party component, you need to dive into the documentation and search for the answer to the question: 'In which directories does the component write files in the container?'

I've always disliked having to spend too much time fiddling with Docker at the initial stage.: I was figuring out how to run containers, from which images to start, creating Makefiles that contained aliases for long Docker commands. I couldn't stand Docker Compose because I didn't want to learn yet another tool in the Docker ecosystem. And to start the containers. This command will bring up 3 containers: it stressed me out, especially if there were more build constructs rather than already built images. All I really wanted was to simply create the product effectively and quickly. But I couldn't sort out the use of Docker at all.

Familiarity with Ansible

Recently (about three months ago), I worked with a DevOps team, almost every member of which had a negative attitude towards Docker. The reasons were:

Docker governs iptables (although it can be disabled in daemon.json)
We won't launch Docker in production.
If the Docker daemon crashes, all containers using the infrastructure will collapse accordingly.
There is no need for Docker.
Why use Docker when we have Ansible and virtual machines?

At the same job, I also got acquainted with another tool — Ansible. I had heard about it before, but had not tried writing my own playbooks. Now I started writing my own tasks and my perspective changed completely! I realized that Ansible has modules to run the same Docker containers, build images, manage networks, etc. Moreover, containers can be deployed not only locally, but also on remote servers! My excitement knew no bounds — I found a PROPER tool and discarded my Makefile and docker-compose files, which were replaced by YAML tasks. The code was reduced by using constructs like loop, block, etc.

Docker for running external components like databases.

Recently, I got familiar with SSH tunneling. It turned out to be very easy to 'forward' a remote server's port to a local port. The remote server can be either a cloud machine or a virtual machine running in VirtualBox. If I or my colleague need a database (or some other external component), we can simply launch a server with that component and shut it down when it's no longer needed. Port forwarding provides the same effect as having a database running in a Docker container.

This command forwards my local port to the remote server with PostgreSQL:

ssh -L 9000:localhost:5432 user@example.com

Using a remote server solves the problem of team development. Several developers can use such a server at the same time, and they don't need to know how to configure PostgreSQL, deal with Docker, or other complexities. The same database can be installed on the remote server in Docker if setting up a specific version is challenging. All that developers will need is SSH access!

I recently read that SSH tunnels are a limited functionality of a regular VPN! You can simply set up OpenVPN or other VPN implementations, configure the infrastructure, and provide it for developers to use. Isn't that so cool!

Fortunately, AWS, Google Cloud, and others offer a year of free usage, so make use of them! They are quite inexpensive if you power them down when not in use. I've always wondered what purposes I would need a remote server like gcloud for; it seems I've found out.

As a virtual machine on a local system, you can use the same Alpine that is actively used in Docker containers. Or other lightweight distributions to speed up machine boot time.

In summary: it's possible and advisable to run databases and other infrastructure utilities on remote servers or in VirtualBox. I don't need Docker for these purposes.

A little about Docker images and distribution.

I've already written. the article In which I wanted to convey that using Docker images does not provide any guarantees. Docker images are only needed to create Docker containers. If you depend on a Docker image, it means you are relying on using Docker containers, and you'll only work with them.

Have you ever seen software developers port their products only in a Docker image?
The majority of products result in binary files for a specific platform; they simply add these to a Docker image derived from the required platform. Haven't you wondered why there are so many similar images on Docker Hub? For instance, search for nginx, and you'll see countless images from different people. These individuals did not develop nginx itself; they simply added the official nginx to their Docker image and complemented it with their configurations for easier container launch.

In general, you can store simply in tgz; if someone needs to run this in Docker, they can just add tgz in the Dockerfile, inherit from the needed environment, and create additional utilities that do not alter the application in tgz. Whoever will create the Docker image will know what this tgz is and what is needed for operation. That's how I use Docker. here

In conclusion: I don't need a Docker registry; I will use some S3 or simply a file storage like Google Drive/Dropbox.

Docker in CI.

All the companies I've worked for are similar to each other. They tend to be product-oriented. That is, they have a single application, one technology stack (or maybe a couple to three programming languages).

These companies use Docker on their servers to run the CI process. The question is—why is it necessary to build projects in a Docker container on their servers? Why not simply prepare an environment for building, for example, by writing an Ansible playbook that installs the required versions of Node.js, PHP, JDK, copies SSH keys, etc., to the server where the build will take place?

Now I realize this is shooting myself in the foot because Docker doesn't bring any profit with its isolation. Here are the CI problems in Docker that I encountered:

You still need a Docker image for the build. You have to find an image or write your own Dockerfile.
There is a 90% chance you'll need to pass some SSH keys and secret data, which you don't want to include in the Docker image.
The container is created and dies, losing all caches with it. The next build will download all project dependencies again, which is slow and inefficient. Time is money.

Developers do not build projects in Docker containers (I used to be a big fan, funny to think about my past self xD). In Java, it's possible to have multiple versions and switch to the needed one with a single command. The same goes for Node.js, there's NVM.

Output

I believe Docker is a very powerful and flexible tool, which is also its downside (sounds strange, right?). Companies easily get ‘hooked’ on it, using it where needed and where not. Developers run their containers, their own environments, and then everything smoothly transitions into CI and production. The DevOps team writes some bicycles to run these containers.

Use Docker only at the very latest stage of your workflow, don't bring it into the project at the beginning. It won't solve your business problems. It will only shift problems to a DIFFERENT level and present its own solutions, making you do double work.

When Docker is necessary: I've come to realize that Docker is excellent at optimizing established processes but not at building basic functionality.

If you do decide to use Docker, then:

be extremely cautious
don't impose Docker usage on developers
localize its usage in one place, do not spread Dockerfile and docker-compose across all repositories

PS:

I recently stumbled upon packer They say it works very well with Ansible and allows for the unification of the image-building process (including docker images).
Also about docker, interesting article.

Thank you for reading to the end; I wish you transparent solutions in your business and productive workdays!

Source: habr.com