Only a person who does not have any external connections with the outside world has not heard about big data today. On HabrΓ©, the topic of Big Data analytics and related topics are popular. But for non-specialists who would like to devote themselves to studying Big Data, it is not always clear what prospects this area has, where Big Data analytics can be applied and what a good analyst can count on. Let's try to figure it out.
Every year the amount of information generated by man is increasing. By 2020, the amount of data stored will increase to 40-44 zettabytes (1 ZB ~ 1 billion GB). By 2025, up to about 400 zettabytes. Accordingly, the management of structured and unstructured data using modern technologies is an area that is becoming increasingly important. Both individual companies and entire states are interested in big data.
By the way, it was in the process of discussing the information boom and ways of processing human-generated data that the term Big Data arose. It is believed that it was first proposed in 2008 by the editor of Nature, Clifford Lynch.
Since then, the Big Data market has been growing by several tens of percent annually. And this trend, according to experts, will continue. Yes, according to the company in 2021, the total volume of the global big data analytics market will increase to $67,2 billion. The annual growth will be about 35,9%.
Why do we need big data analytics?
It allows you to identify extremely valuable information from structured or unstructured data sets. Thanks to this, a business, for example, can identify trends, predict production performance and optimize its own costs. It is clear that in order to reduce costs, companies are ready to implement the latest solutions.
Technologies and methods of analysis that are used to analyze Big Data:
- data mining;
- crowdsourcing;
- data mixing and integration;
- machine learning;
- artificial neural networks;
- pattern recognition;
- predictive analytics;
- simulation modeling;
- spatial analysis;
- statistical analysis;
- visualization of analytical data.
Analytics Big Data in the world
Now big data analytics is used in more than 50% of companies around the world. Despite the fact that in 2015 this figure was only 17%. Big Data is most actively used by companies that work in the field of telecommunications and financial services. Then there are companies that specialize in technology in healthcare. Minimal use of Big Data analytics in educational companies: in most cases, representatives of this field have declared their intention to use technology in the near future.
In the United States, Big Data analytics is used most actively: more than 55% of companies from various fields work with this technology. In Europe and Asia, the demand for big data analytics is not much lower - about 53%.
And what about Russia?
According to IDC analysts,. The growth of the market for such solutions in Central and Eastern Europe is quite active, every year this figure increases by 11%. By 2022, it will reach $5,4 billion in quantitative terms.
In many ways, such a rapid development of the market is due to the growth of this sector in Russia. In 2018, revenue from the sale of relevant solutions in the Russian Federation amounted to 40% of the total investment in Big Data processing technologies for the entire region.
In the Russian Federation, companies from the banking and public sectors, the telecommunications industry and industry spend the most on Big Data processing.
What does Big Data Analyst do and how much does he earn in Russia?
The Big Data Analyst is responsible for examining vast amounts of information, both semi-structured and unstructured. For banking organizations, these are transactions, for operators - calls and traffic, in retail - customer visits and purchases. As mentioned above, Big Data analysis allows you to discover links between various factors of the "raw information history", for example, a production process or a chemical reaction. Based on the analysis data, new approaches and solutions are being developed in various fields - from production to medicine.
Skills required for a Big Data analyst:
- The ability to quickly understand the features in the area for which the analysis is carried out, to immerse yourself in the aspects of the desired area. It can be retail, oil and gas industry, medicine, etc.
- Knowledge of methods of statistical data analysis, building mathematical models (neural networks, Bayesian networks, clustering, regression, factorial, dispersion and correlation analyses, etc.).
- Be able to extract data from different sources, convert them for analysis, upload them to an analytical database.
- Proficiency in SQL.
- Knowledge of English at a level sufficient for trouble-free reading of technical documentation.
- Knowledge of Python (at least the basics), Bash (it is very difficult to do without it in the process of work), plus it is desirable to know the basics of Java and Scala (required for the active use of Spark, one of the most popular frameworks for working with big data).
- Ability to work with Hadoop.
Well, how much does a Big Data analyst get?
Big Data specialists are now in short supply, demand exceeds supply. This is because business is coming to an understanding: development requires new technologies, and the development of technologies requires specialists.
So, Data Scientist and Data Analytic in the USA according to the recruitment agency Glassdoor. The average salary of these specialists in America starts from $100 per year.
In Russia, machine learning specialists receive from 130 to 300 rubles a month, big data analysts - from 73 to 200 rubles a month. It all depends on experience and qualifications. Of course, there are vacancies with a lower salary, there are with a higher one. Maximum demand for big data analysts in Moscow and St. Petersburg. Not surprisingly, Moscow accounts for about 50% of active vacancies (according to hh.ru). Much less demand is in Minsk and Kyiv. It is worth noting that some vacancies offer flexible hours and remote work. But in general, companies need specialists who work in the office.
Over time, we can expect an increase in demand for Big Data analysts and representatives of related specialties. As mentioned above, no one has canceled the personnel shortage in the field of technology. But, of course, in order to become a Big Data analyst, you need to study and work, improving both the skills listed above and additional ones. One of the opportunities to start the path of Big Data analytics is and try your hand at working with big data.
Source: habr.com