How I ran Docker inside Docker and what came of it

Hi all! In his previous article, I promised to talk about running Docker in Docker and about the practical aspects of using this lesson. It's time to fulfill your promise. An experienced devops, perhaps, will object that those who need Docker inside Docker simply forward the Docker daemon socket from the host into the container and this is enough in 99% of cases. But do not rush to throw cookies at me, because we will talk about the real run of Docker inside Docker. This solution has many possible applications, and this article is about one of them, so sit back and straighten your arms in front of you.

Home

It all started on a rainy September evening when I was cleaning out a $5 machine rented from Digital Ocean, which was hanging dead because Docker filled up all 24 gigabytes of available disk space with its images and containers. The irony was that all these images and containers were transient and were only needed to test the performance of my application every time a new version of a library or framework was released. I tried writing shell scripts and setting up a cron schedule for cleaning up garbage, but it didn’t help: every time everything inevitably ended up with my server’s disk space being eaten up and the server freezing (at best). At some point, I came across an article about how to run Jenkins in a container and how it can create and delete assembly pipelines through a docker daemon socket forwarded to it. I liked the idea, but I decided to go ahead and try experimenting with directly running Docker inside Docker. It seemed to me then a completely logical decision to download docker images and create containers for all the applications that I need to test inside another container (let's call it a staging container). The idea was to run a staging container with the -rm flag, which automatically deletes the entire container with all its contents when it stops. I poked around with the docker image from docker itself (https://hub.docker.com/_/docker), but it turned out to be too cumbersome and I never managed to get it to work as I needed and I wanted to go all the way myself.

Practice. cones

I set out to make the container work as I needed and continued my experiments, which resulted in a myriad of cones. The result of my self-torture was the following algorithm:

1. Run the Docker container in interactive mode.

   docker run --privileged -it docker:18.09.6

   Pay attention to the version of the container, step to the right or to the left and your DinD turns into a pumpkin. In fact, things break quite often with the release of a new version.
   We must immediately get into the shell.

2. We are trying to find out which containers are running (Answer: none), but let's run the command anyway:

   docker ps

   You will be a little surprised, but it turns out that the Docker daemon is not even running:

   error during connect: Get http://docker:2375/v1.40/containers/json: dial tcp: lookup docker on 
   192.168.65.1:53: no such host

3. Let's run it on our own:

   dockerd &

   Another unpleasant surprise:

   failed to start daemon: Error initializing network controller: error obtaining controller instance: failed 
   to create NAT chain DOCKER: Iptables not found

4. Install the iptables and bash packages (everything is more pleasant to work in bash than in sh):

   apk add --no-cache iptables bash

5. We start bash. Finally we are back in the usual shell

6. Let's try running Docker again:

   dockerd &

   We should see a long sheet of logs ending with:

   INFO[2019-11-25T19:51:19.448080400Z] Daemon has completed initialization          
   INFO[2019-11-25T19:51:19.474439300Z] API listen on /var/run/docker.sock

7. Press Enter. We are back in the tower.

From now on we can try to run other containers inside our Docker container, but what if we want to start another Docker container inside our Docker container or something goes wrong and the container crashes? Start all over.

Own DinD container and new experiments

In order not to repeat the above steps again and again, I created my own DinD container:

https://github.com/alekslitvinenk/dind

The working DinD solution gave me the ability to run Docker inside Docker recursively and do more daring experiments.
One such (successful) experiment with running MySQL and Nodejs is what I'm about to describe.
The most impatient can see how it was here

So, let's begin:

1. Run DinD interactively. In this version of DinD, we need to manually map all the ports that our child containers can use (I'm already working on this)

   docker run --privileged -it 
   -p 80:8080 
   -p 3306:3306 
   alekslitvinenk/dind

   We get into the bash, from where we can immediately start launching child containers.

2. Start MySQL:

   docker run --name mysql -e MYSQL_ROOT_PASSWORD=strongpassword -d -p 3306:3306 mysql

3. We connect to the database in the same way as we would connect to it locally. We make sure everything works.

4. We start the second container:

   docker run -d --rm -p 8080:8080 alekslitvinenk/hello-world-nodejs-server

   Please note that the port mapping here will be exactly 8080:8080, since we have already mapped port 80 from the host to the parent container on port 8080.

5. We go to localhost in the browser, make sure that the server responds with "Hello World!".

In my case, the experiment with nested Docker containers turned out to be quite positive and I will continue to develop the project and use it for staging. It seems to me that this is a much more lightweight solution than the same Kubernetes and Jenkins X. But this is my subjective opinion.

I think that's all for today's article. In the next article, I will describe in more detail experiments with running Docker recursively in Docker and mounting directories deep into nested containers.

PS If you find this project useful, then please give it a star on GitHub, fork it and tell your friends.

Edit1 Fixed bugs, made focus on 2 videos

Source: habr.com