Building your first DevOps chain in five steps for beginners.
DevOps has become a panacea for too slow, disconnected and otherwise problematic development processes. But you need minimal knowledge in DevOps. It will cover concepts such as the DevOps chain and how to create one in five steps. This is not a complete guide, but only a "fish" that can be expanded. Let's start with history.
My introduction to DevOps
I used to work with clouds at Citi Group and develop an IaaS web application to manage Citi's cloud infrastructure, but I've always been interested in how to optimize the development chain and improve the culture among developers. Greg Lavender, our CTO for Cloud Architecture and Infrastructure, recommended this book to me. . It explains DevOps principles beautifully and reads like a novel.
The table on the back shows how often companies roll out new versions:
How do Amazon, Google and Netflix manage to roll out so much? And it's simple: they figured out how to create an almost perfect DevOps chain.
Things were very different for us at Citi until we switched to DevOps. Then my team had different environments, but we did the delivery to the development server manually. All developers had access to only one development server based on IBM WebSphere Application Server Community Edition. With a simultaneous attempt to deliver, the server βfellβ, and each time we had to βpainfullyβ negotiate among ourselves. We also had insufficient code coverage with tests, a time-consuming manual delivery process, and no way to track the delivery of code with the help of some task or client requirement.
It was clear that something urgently needed to be done, and I found a like-minded colleague. We decided to create the first DevOps chain together - he set up a virtual machine and a Tomcat application server, and I took care of Jenkins, integration with Atlassian Jira and BitBucket, as well as code coverage with tests. The project was successful: we fully automated the development chain, achieved almost 100% uptime on the development server, were able to monitor and improve code coverage with tests, and a Git branch could be tied to a Jira delivery and issue. And almost all of the tools we used to build the DevOps chain were open source.
In fact, the chain was simplified, because we did not even apply advanced configurations using Jenkins or Ansible. But we succeeded. Perhaps this is a consequence of the principle (aka the 80/20 rule).
A Brief Description of the DevOps and CI/CD Chain
DevOps has different definitions. DevOps, like Agile, includes different disciplines. But most will agree with the following definition: DevOps is a method, or life cycle, of software development, the main principle of which is to create a culture where developers and other employees are βon the same wavelengthβ, manual labor is automated, everyone does what they are best at, the frequency of deliveries increases, the productivity of work increases, flexibility increases.
While tools alone are not enough to create a DevOps environment, they are indispensable. The most important of these is continuous integration and continuous delivery (CI/CD). There are different stages in the chain for each environment (e.g. DEV (development), INT (integration), TST (testing), QA (quality assurance), UAT (user acceptance testing), STG (preparation), PROD (use)), manual tasks are automated, developers can make quality code, deliver it, and can easily rebuild.
This note describes how to create a DevOps chain in five steps, as shown in the picture below, using open source tools.
Let's get down to business.
Step 1: CI/CD Platform
First of all, you need a CI/CD tool. Jenkins is an MIT-licensed, open-source CI/CD tool written in Java that popularized the DevOps movement and has become the de facto standard for CICD.
What is Jenkins? Imagine that you have a magical control panel for a variety of services and tools. On its own, a CI/CD tool like Jenkins is useless, but with different tools and services, it becomes all-powerful.
In addition to Jenkins, there are many other open source tools, choose any.
Here's what a DevOps process looks like with a CI/CD tool
You have a CI/CD tool in localhost, but there isn't much to do yet. Let's move on to the next step.
Step 2: Versioning
The best (and arguably easiest) way to test the magic of a CI/CD tool is to integrate it with a source control management (SCM) tool. Why do you need version control? Let's say you're making an application. You write it in Java, Python, C++, Go, Ruby, JavaScript, or any other language that's a wagon and a little cart. What you write is called source code. At first, especially if you're working alone, you can save everything to a local directory. But as the project grows and more people join, you need a way to share code changes but avoid conflicts when merging changes. And you also need to somehow restore previous versions without using backups and using the copy-paste method for code files.
And here without SCM anywhere. The SCM stores code in repositories, manages versions of it, and coordinates it among developers.
There are many SCM tools, but Git has deservedly become the de facto standard. I advise you to use it, but there are other options.
Here is what the DevOps pipeline looks like after adding the SCM.
The CI/CD tool can automate source code upload and download and team collaboration. Not bad? But now how to make a working application from this, loved by billions of users?
Step 3: Build Automation Tool
Everything is going as it should. You can upload code and commit changes to source control, and invite friends to work with you. But you don't have an app yet. For this to be a web application, it must be compiled and packaged for distribution or run as an executable. (An interpreted programming language like JavaScript or PHP does not need to be compiled.)
Use a build automation tool. Whichever tool you choose, it will assemble the code in the right format and automate cleanup, compilation, testing, and delivery. Build tools vary by language, but the following open source options are commonly used.
Perfect! Now let's insert the build automation tool configuration files into source control so that the CI/CD tool builds them.
It feels good. But where is all this to roll out now?
Step 4: Web Application Server
So, you have a packaged file that can be executed or rolled out. For an application to be really useful, it must have some kind of service or interface, but you need to put it all somewhere.
A web application can be hosted on a web application server. The application server provides an environment where you can execute packaged logic, render interfaces, and expose web services over a socket. You need an HTTP server and a few other environments (a virtual machine, for example) to install the application server. For now, let's pretend you're dealing with all of this as you go (although I'll talk about containers below).
There are several open web application servers.
We already have an almost working DevOps chain. Great job!
In principle, you can stop here, then you can handle it yourself, but itβs worth talking about the quality of the code.
Step 5: Test coverage
Testing takes a lot of time and effort, but it's better to find bugs right away and improve the code to please end users. For this purpose, there are many open tools that will not only test the code, but also advise on how to improve it. Most CI/CD tools can plug into these tools and automate the process.
Testing is divided into two parts: testing frameworks for writing and executing tests, and tools with hints to improve code quality.
Testing Frameworks
Tools with quality tips
Most of these tools and frameworks are written for Java, Python, and JavaScript because C++ and C# are proprietary (although GCC is open source).
We have applied the test coverage tools, and now the DevOps pipeline should look like the picture at the beginning of the tutorial.
Additional steps
Containers
As I said before, an application server can be hosted in a virtual machine or a server, but containers are more popular.
? In short, in a virtual machine, the operating system often takes up more space than the application, and a container usually suffices with a few libraries and configuration. In some cases, virtual machines are indispensable, but the container can accommodate the application along with the server at no extra cost.
For containers, Docker and Kubernetes are usually taken, although there are other options.
Read articles about Docker and Kubernetes at :
Middleware automation tools
Our DevOps chain is focused on collaborative building and delivery of an application, but there are other interesting things you can do with DevOps tools. For example, use Infrastructure as Code (IaC) tools, also known as middleware automation tools. These tools help automate installation, management, and other tasks for middleware. For example, an automation tool can take applications (web application server, database, monitoring tools) with the correct configurations and push them to the application server.
Here are some options for open middleware automation tools:
Details in the articles :
And now what?
This is just the tip of the iceberg. The DevOps chain can do much more. Start with a CI/CD tool and see what else you can automate to make your job easier. Don't forget about for effective collaboration.
Here are some more good DevOps articles for beginners:
You can also integrate DevOps with open agile tools:
Source: habr.com