Many foreign authors agree that the era of Big Data has come to an end. In this case, the term Big Data refers to technologies based on Hadoop. Many authors can even confidently name the date when Big Data left this world, which is June 5, 2019.
What happened on this significant day?
On this day, MAPR announced it would cease operations if it could not find funds to continue. Later, in August 2019, MAPR was acquired by HP. However, returning to June, one cannot overlook the tragic nature of this period for the Big Data market. That month saw a significant drop in the stock prices of CLOUDERA, a leading player in the market, which had merged with the chronically unprofitable HORTOWORKS in January of the same year. The decline was substantial at 43%, ultimately reducing CLOUDERA's market capitalization from $4.1 billion to $1.4 billion.
It is impossible not to mention that rumors of a bubble inflating in technologies based on Hadoop had been circulating since December 2014, yet it bravely endured for nearly five more years. These rumors were based on Google's rejection of its invention, the Hadoop technology. However, the technology took root during the transition of companies to cloud processing tools and the rapid development of artificial intelligence. Thus, looking back, one can confidently say that its demise was anticipated.
Thus, the era of Big Data has come to an end, but in the process of working with big data, companies realized all the nuances of handling it, the benefits that Big Data can bring to business, and also learned to use artificial intelligence to extract value from raw data.
This makes the question of what will replace this technology and how analytics technologies will develop even more interesting.
Augmented analytics
During the events described, companies operating in the data analytics sphere were not idle. This can be judged based on the information about transactions that took place in 2019. In the current year, the largest market deal was made – the acquisition of the Salesforce analytics platform Tableau for $15.7 billion. A smaller deal occurred between Google and Looker. And of course, we cannot overlook the acquisition of the big data platform Attunity by Qlik.
Market leaders in BI and Gartner specialists have announced a significant shift in approaches to data analysis that will completely disrupt the BI market and lead to the replacement of BI with AI. In this context, it is important to note that the abbreviation AI does not stand for 'Artificial Intelligence' but 'Augmented Intelligence.' Let's take a closer look at what is hidden behind the term 'Augmented Analytics.'
Augmented analytics, like augmented reality, is based on several common postulates:
- the ability to communicate using NLP (Natural Language Processing), i.e., in human language;
- the use of artificial intelligence, meaning that the data will be pre-processed by machine intelligence;
- and, of course, recommendations available to the system user that are generated by artificial intelligence.
According to analytics platform manufacturers, their use will be available to users without special skills, such as knowledge of SQL or similar scripting languages, without statistical or mathematical training, and without knowledge of popular languages specialized in data processing and corresponding libraries. Such individuals, referred to as 'Citizen Data Scientists,' need only exceptional business qualifications. Their task is to capture business insights from the hints and forecasts provided by artificial intelligence, while fine-tuning their assumptions using NLP.
Describing the user interaction process with systems of this class, one can envision the following scenario. A person arrives at work and launches the relevant application, along with the usual set of reports and dashboards that can be analyzed using standard approaches (sorting, grouping, performing arithmetic actions), sees certain prompts and recommendations, something like: "To achieve your sales KPI, you should apply a discount on products in the 'Gardening' category." In addition, the person can turn to the corporate messenger: Skype, Slack, etc. They can ask the bot questions, either in text or voice: "Show me the five most profitable clients." Upon receiving the appropriate response, they must make an optimal decision based on their business experience and generate profit for the company.
If we take a step back and look at the composition of the analyzed information, even at this stage, augmented analytics products can simplify life for people. Ideally, a user only needs to specify the sources of the desired information to the analytical product, and the program itself will take care of creating the data model, linking tables, and similar tasks.
All of this should, first and foremost, ensure the 'democratization' of data, meaning that anyone can engage in analyzing the entire array of information available to the company. The decision-making process should be supported by statistical analysis methods. Access time to data should be minimal, eliminating the need to write scripts and SQL queries. And of course, it will allow for savings on high-salaried Data Science specialists.
Hypothetically, technologies open up very promising prospects for business.
What Replaces Big Data
However, I actually began my article with Big Data. To expand on this topic, I could not avoid a brief excursion into modern BI tools, which are often based on Big Data. The fate of big data is now clearly predetermined, and that is cloud technologies. I emphasized the deals made with BI producers to demonstrate that now every analytical system is underpinned by cloud storage, and cloud services feature BI as the front end.
While not forgetting pillars in the database sphere such as ORACLE and Microsoft, it is important to note their chosen direction for business development is cloud computing. All offered services can be found in the cloud, but some cloud services are no longer available on-premise. Significant work has been done to utilize machine learning models, libraries have been created for users, and interfaces have been configured for convenience, from model selection to startup timing.
Another important advantage of using cloud services, which manufacturers highlight, is the availability of virtually unlimited datasets on any topic for training models.
However, the question arises: how well will cloud technologies take root in our country?
Source: habr.com
