A few years ago, many organizations viewed DevOps as a promising experiment rather than a mainstream approach to software development. DevOps is now a proven and powerful set of development and deployment practices and tools to speed up new product releases and improve productivity. More importantly, the DevOps effect is aimed at the overall growth of the business and increasing its profitability.
Team translated the most interesting compiled by DevOps Research & Assessment (DORA) experts. The study involved 31 professionals from around the world. Let's take a look at what has changed in the industry in 000 and how businesses can improve the efficiency of software delivery.
How the state of DevOps is affected by industry and company size
The study found no relationship between DevOps performance and the organization's industry, with the exception of retail, where performance was marginally better. This, in particular, is due to the fact that retailers need to quickly respond to fluctuations in demand and customer needs. According to the study, any company can achieve a high level in DevOps, including the financial sector and the public sector.
DevOps performance in companies with more than 5000 employees was lower than in companies with less than 5000 employees. Most likely, this is due to the fact that in large organizations there are larger processes, stricter control, more complex architecture of IT systems, which introduces delays in the process of developing and rolling out code. At the same time, experts believe that the size of the company does not prevent success in building DevOps, it just may require more effort in some cases.
How to assess the level of DevOps in a company
The experts compared the DevOps processes to the benchmark, dividing the survey participants into four groups of best, good, average, and poor.
For the report, four key metrics for evaluating the effectiveness of DevOps were taken: the time it takes to make changes in software development, the frequency of deployment, the frequency of failures, and the recovery time.
Four levels of DevOps - assess where your company is:
Metric for evaluating the effectiveness of software delivery for the main services and applications of the company
Teams with the best records
Teams with good records
Teams with averages
Low performing teams
Deployment frequency
How often the company deploys code to production or releases it to end users.
On demand, multiple deployments per day
Once a day to once a week
From once a week to once a month
Once a month / several months
Change execution time
How long does it take to go from test to software running successfully in production.
Less than a day
From one day to a week
From one week to a month
One month to six months
Service recovery time
How long does it take to restore the service after an incident or bug that affects users.
Less than an hour
During the day
In a week
From a week to a month
Change failure rate
What percentage of updates or new releases result in degraded service and require fixes.
0-15%
0-15%
0-15%
46-60%
The study revealed the following trend: the number of teams with a high level of performance almost tripled, increasing from 7% of all respondents in 2018 to 20% in 2019.
The distribution of development teams by performance levels.
Compared to the teams in the low performing group, the high performing DevOps teams:
- Completed 208 times more code deployments.
- 106 times less time spent deploying code.
- 7 times less likely to encounter failures.
- 2,604 times faster software recovery after failures.
In addition, high-performing DevOps teams are twice as likely to meet or exceed their organizational KPIs as low-performing teams.
Many experts think that it is impossible to achieve an increase in all indicators at the same time, it is necessary to compromise. For example, some believe that an increase in the speed of rolling out releases can negatively affect the reliability of the software delivery process and the provision of services. However, studies have shown that speed and stability of results are not mutually exclusive.
I donβt see anything surprising in the growth in the number of DevOps teams, it is natural: the DevOps philosophy is now popular, and the number of startups is growing.
But, in my opinion, the experts chose not entirely correct parameters for evaluating the effectiveness of DevOps.
Evaluating it by the speed of rolling out the code is at least strange. This only applies to startups where the key parameter will be the speed of bringing the product to market, and often the product is launched in raw form. Under such conditions, mechanisms that accelerate development and delivery to production are vital. But for established software, such as financial or medical, the failure rate parameter may not exist - failures may be unacceptable.
Similarly, with the service recovery time: for any developed service, it should be calculated in seconds, and for many services, downtime is unacceptable, for this they came up with seamless rollout technologies (for example, green / blue).
Also, do not focus on the number of code deployments - it depends on the need and competencies of the development team. If a deployment is about adding new functionality, that's one thing, but if it's about fixing bugs from previous deployments, it's completely different.
Denis Romanenko, freelance expert at Mail.ru Cloud Solutions
How to improve DevOps processes
The report presents two areas that will help improve DevOps: improving the efficiency of software development and delivery, and improving workforce productivity.
Each of the directions includes its own components, improving which, you can achieve the desired goal.
According to the report, the key to digital transformation is corporate culture. High-performing DevOps teams need a culture of trust and safety, understanding of performance, and clear goals. This environment allows team members to make informed decisions, speak their minds, and be more creative.
Cloud technologies, continuous delivery, disaster recovery testing, change management will also help improve the efficiency of software development and delivery. Productivity can be increased by investing in easy-to-use tools, reducing technical debt, i.e. reducing the percentage of inefficient code and obsolete technologies, organizing an enterprise knowledge base and access to external solutions.
I think that the methodology and ideology of DevOps is precisely that these processes do not depend on external conditions, such as the cloud or your own hardware. The cloud itself is nothing more than a tool, somewhere it will help, somewhere it will hinder or will not be in demand.
Denis Romanenko, freelance expert at Mail.ru Cloud Solutions
Below, we will look at some of the components of improving the efficiency of DevOps teams.
Cloud computing drives DevOps success
In 2019, more organizations are choosing cloud-based solutions that significantly increase the productivity of DevOps teams.
What infrastructures are used by DevOps teams.
DORA found that 80% of respondents post . However, only 29% of respondents have implemented all five of the National Institute of Standards and Technology's core cloud computing characteristicsβthe most important standard for evaluating the value of the cloud in DevOps.
Characterization
Percentage of those who used
Self-service on demand
Consumers can automatically provision computing resources
as needed, without the participation of the provider.
57%
(+ 11% since 2018)
Wide network access
Cloud capabilities are available through different platforms,
such as mobile phones, tablets, laptops and workstations.
60%
(+ 14% since 2018)
Resource pool
Provider resources are combined into a multi-tenant model where physical and virtual resources are dynamically assigned on demand.
58%
(+ 15% since 2018)
Scalability and elasticity
Resources scale horizontally or vertically on demand, are virtually unlimited, and can be provisioned in any quantity at any time.
58%
(+135 since 2018)
Transparency
Cloud systems automatically monitor, optimize and report resource usage depending on the type of service: data storage and processing, amount of traffic,
active user accounts.
62%
(+ 14% since 2018)
Platform as a Service (PaaS) is increasingly moving towards a deployment model centered around containers. Cloud platforms make it easy to deploy software, so teams only need to worry about running the application code itself. Scaling, resource planning, administration and maintenance of infrastructure also goes to the side of providers.
For cloud providers, the provision of a variety of services is becoming a universal standard: virtual machine networks, identity and access management (IAM), storage and databases, machine learning, the Internet of things (IoT), container solutions, security solutions and others.
Cloud provider customers pay only for the resources they use, which ensures transparency of costs, unlike traditional data centers, where development cost information is difficult or impossible to obtain. Respondents from companies that meet the cloud characteristics listed above are 2,6 times more likely to estimate the cost of software operation, 2 times more likely to understand which applications require more resources, 1,65 times more likely to stay within the budget allocated to IT.
Sometimes it turns out that hiring a competent specialist and taking the allocated capacities in the data center is more profitable than paying for the cloud. Which option is better depends on the profile and scale of the company, the availability of its own staff of IT specialists and expertise. For example, it is convenient to use the cloud at the start of a business or if the company does not have its own IT department. When scaling, it may be more cost-effective to keep all or part of the infrastructure on-premise.
Denis Romanenko, freelance expert at Mail.ru Cloud Solutions
DevOps technical practices
Many organizations looking to adopt DevOps are looking for a set of guidelines or best practices. However, there are no identical companies, so which practices to choose depends on the current state of the business and its goals.
That said, there are general areas that help improve DevOps performance: some of them are developed at the team level, others require effort at the organization level.
What areas of growth are highlighted for DevOps teams in 2019:
At the organization level
- loosely coupled architecture
- implementation of changes
- code support
Team level
- continuous integration
- test automation
- deployment automation
- monitoring
- development pipeline
At the team and organization level
- use of cloud services
- disaster recovery testing
The study confirmed the positive impact of loosely coupled architecture on the effectiveness of DevOps.
A loosely coupled architecture is when teams can independently test, deploy, and modify systems on demand, independent of other teams, with no additional support, resources, approval, or less feedback. This allows you to work more efficiently, but requires a high level of organization and management.
This approach is possible only for startups and with some reservations. Other companies may have a different situation. A good example is banking/fintech. Only proprietary solutions can be used there, but DevOps practices will be applied.
Denis Romanenko, freelance expert at Mail.ru Cloud Solutions
Successful DevOps teams automate everything
allows you to bring services and applications to production with less cost and risk, as well as support releases in accordance with the goals of the organization.
Successful CI/CD also means that teams can push changes into production on demand and have immediate feedback on the quality of the deployment that can be quickly worked out and improved on for the next deployment cycle.
The report shows that successful DevOps teams invest in a wide range of supporting processes, practices, and tools:
- 92% use automated assembly tools;
- 87% use automated unit tests;
- 57% extend automation to acceptance testing;
- 72% automate test deployments, 69% do the same for production deployments
- 69% integrate chatbots into their deployment process
- 57% integrate with monitoring tools.
It is important to choose the right tools and technologies
When building complex systems and managing business-critical infrastructures, it is important to choose technologies:
- which are easy to use both at the first connection and in permanent operation;
- that help you achieve your goals.
The report explored the tools used to deploy software via CI/CD and test automation tools - these are the technologies that underlie DevOps.
What technologies are used by DevOps teams:
Technologies
Low performing teams
Teams with averages
Teams with good records
High performing teams
Combination of proprietary, open source and commercial boxed products
30%
34%
32%
33%
Mostly open source and highly customized out-of-the-box solutions
17%
8%
7%
10%
Mostly open source and boxed solutions with little customization
14%
21%
18%
20%
Primarily boxed commercial solutions
8%
12%
8%
4%
Internal developments and proprietary solutions for the company
20%
6%
5%
6%
Primarily open source with strong customization
6%
7%
5%
12%
Primarily open source with little customization
5%
12%
24%
15%
Tool friendliness has a significant impact on a team's ability to maximize the value of their chosen technology stack: engineers with easy-to-use technologies are 1,5 times more likely to belong to high-performing teams.
In my opinion, from this table one gets the feeling that in order to be a successful DevOps team, you need to follow the fashion, not the technical task.
A competent specialist chooses tools for the task, and not vice versa. To solve any problem, there are always several tools and approaches. A specific tool is determined by: the specifics of the task; how familiar is the staff with this tool (how high is the entry threshold, if the tool is new); financial component, if any.
Denis Romanenko, freelance expert at Mail.ru Cloud Solutions
Disaster Recovery
Every organization that depends on the operation of the software must have . The report shows which types of resiliency testing different companies use.
What Types of Testing Companies Use for Disaster Recovery
Type of test
Low performing teams
Teams with averages
Teams with good records
High performing teams
The average
Tests that do not affect real systems
35%
26%
27%
30%
28%
Infrastructure failover (including data center)
27%
43%
34%
38%
38%
Application failure testing
25%
46%
41%
49%
43%
Simulation of incidents with violation of test systems
18%
22%
23%
29%
23%
Modeling incidents with disruption of work systems
18%
11%
12%
13%
12%
Creating automation and systems that disrupt
production systems on a regular, ongoing basis
9%
8%
7%
9%
8%
Only 40% of respondents conduct disaster recovery testing annually using one or more of these methods. At the same time, companies that conduct disaster recovery tests have a higher level of service availability. The report shows that high-performing DevOps teams are 1.4x more likely to incorporate disaster recovery test data into software development and deployment processes.
It is important to provide DevOps teams with access to information
Keeping DevOps teams productive can help you easily find information to solve problems. This is especially true in today's technological environment, which consists of complex systems.
The sources of such information can be divided into two groups:
- Internal sources: company documentation on creating and maintaining code, corporate knowledge bases, repositories and more. Those DevOps teams that used internal knowledge sources were 1,73 times more productive.
- External sources: search engines and stack replenishment. Outsourced DevOps teams were 1,67 times more productive. External technologies provide a great advantage for learning and growth, especially the use of public clouds and open source tools.
It is important for companies to reduce technical debt
Technical debt includes code or systems with known bugs that have not been fixed; insufficient test coverage; low quality code or design; artifacts that are not used but not removed; implementations that the team cannot effectively support; outdated technologies; incomplete or outdated documentation.
Experts have found that technical debt has a negative impact on the effectiveness of DevOps. Teams with high technical debt were 1,6 times less productive. High-performing teams were 1,4 times more likely to have low technical debt.
Key Findings from the State of DevOps Research
- The percentage of high-performing DevOps teams has almost tripled to 20%. This means that businesses understand the promise of improving software development and delivery practices, companies are actively implementing DevOps in their IT departments.
- The rapid delivery of applications and services is at the heart of technology and organizational transformation. The speed and stability of release rollout increases profits and customer satisfaction.
- Cloud computing continues to be key to high performance DevOps teams. The use of clouds allows you to organize the delivery of software at the right speed, ensures the availability, scalability and performance of the infrastructure.
- The effectiveness of DevOps teams can be improved by paying attention to the productivity of team members, providing a comfortable psychological atmosphere and using convenient tools.
- Increasing the speed of rolling out releases with the right approach does not affect the stability of the company's services and applications.
Source: habr.com