Many foreign authors agree that the era of Big Data has come to an end. And in this case, the term Big Data refers to technologies based on Hadoop. Many authors can even confidently name the date when Big Data left this world and this date is 05.06.2019/XNUMX/XNUMX.
What happened on this momentous day?
On this day, the MAPR company promised to suspend its work if it could not find funds for further operation. Later, in August 2019, MAPR was acquired by HP. But returning to June, one cannot fail to note the tragedy of this period for the Big Data market. This month there was a collapse in the stock prices of CLOUDERA, a leading player in the indicated market, which merged with the chronically unprofitable HORTOWORKS in January of the same year. The collapse was very significant and amounted to 43%, in the end, the capitalization of CLOUDERA decreased from 4,1 to 1,4 billion dollars.
It's impossible not to say that the Hadoop-based tech bubble has been rumored since December 2014, but it bravely held out for nearly five more years. These rumors were based on the refusal of Google, the company in which Hadoop technology was born, from its invention. But the technology has taken root during the transition of companies to cloud computing and the rapid development of artificial intelligence. Therefore, looking back, it is safe to say that death was expected.
Thus, the era of Big Data has come to an end, but in the process of working on big data, companies have realized all the nuances of working on them, the benefits that Big Data can bring to business, and have also learned how to use artificial intelligence to extract value from raw data.
The more interesting is the question of what will replace this technology and how analytics technologies will develop further.
Augmented Analytics
During the events described, companies working in the field of data analysis did not sit still. What can be judged based on information about transactions that occurred in 2019. This year, the largest transaction in the market was carried out - the acquisition of the Tableau analytics platform by Salesforce for $15,7 billion. A smaller deal took place between Google and Looker. And of course, one cannot fail to note the acquisition by Qlik - the big data of the Attunity platform.
BI market leaders and Gartner specialists announce a grand shift in approaches to data analysis, this shift will completely destroy the BI market and lead to the replacement of BI with AI. In this context, it should be noted that the abbreviation AI is not "Artificial intelligence" but "Augmented Intelligence". Let's take a closer look at what lies behind the words "Augmented Analytics".
Augmented analytics, like augmented reality, is based on several general postulates:
- the ability to communicate using NLP (Natural Language Processing), i.e. in human language;
- the use of artificial intelligence, which means that the data will be pre-processed by machine intelligence;
- and of course, the recommendations available to the user of the system, which were just generated by artificial intelligence.
According to the manufacturers of analytical platforms, their use will be available to users who do not have special skills, such as knowledge of SQL or a similar scripting language, who do not have statistical or mathematical training, who do not have knowledge in the field of popular languages ββspecializing in data processing and related libraries. Such people, called "Citizen Data Scientist", should only have outstanding business qualifications. Their task is to capture business insights from the clues and predictions that artificial intelligence will give them, and they will be able to refine their guesses using NLP.
Describing the process of users working with systems of this class, one can imagine the following picture. A person, coming to work and launching the corresponding application, in addition to the usual set of reports and dashboards that can be analyzed using standard approaches (sorting, grouping, performing arithmetic operations), sees certain tips and recommendations, something like: βIn order to achieve KPI, according to number of sales, you should apply the discount on products from the Gardening category. In addition, a person can turn to a corporate messenger: Skype, Slack, etc. Can ask the robot questions, text or voice: "Bring me the five most profitable customers." Having received the appropriate answer, he must make the best decisions based on his experience in business and bring profit to the company.
If you take a step back and look at the composition of the analyzed information, and at this stage, augmented analytics class products can make life easier for people. Ideally, it is assumed that the user only needs to point the analytical product to the sources of the desired information, and the program itself will take care of creating a data model, linking tables, and similar tasks.
All this should, first of all, ensure the βdemocratizationβ of data, i.e. anyone can analyze the entire array of information available to the company. The decision-making process should be supported by methods of statistical analysis. Data access time should be minimal, so there is no need to write scripts and SQL queries. And of course, it will be possible to save on highly paid Data Science specialists.
Hypothetically, technology opens up very bright prospects for business.
What replaces Big Data
But, in fact, I started my article with Big Data. And I could not develop this topic without a brief digression into modern BI tools, the basis for which, often, is Big Data. The fate of big data is now clearly sealed, and that is cloud computing. I focused on deals made with BI vendors to demonstrate that now every analytics system has cloud storage under it, and cloud services have BI as a front end.
Without forgetting such pillars in the field of databases as ORACLE and Microsoft, it is necessary to note the direction of business development they have chosen and this cloud. All services offered can be found in the cloud, but some cloud services are no longer available on-premise. They have done significant work on the use of machine learning models, created libraries available to users, configured interfaces for the convenience of working with models from its selection to setting the start time.
Another important advantage of using cloud services, which is voiced by manufacturers, is the availability of almost unlimited data sets on any topic for training models.
However, the question arises, how cloud technologies will take root in our country?
Source: habr.com