There are hundreds of articles on the Internet about the benefits of analyzing customer behavior. Most often this applies to the retail sector. From the analysis of grocery baskets, ABC and XYZ analysis to retention marketing and personal offers. Various techniques have been used for decades, the algorithms are thought out, the code is written and debugged - take it and use it. In our case, one fundamental problem arose - we at ISPsystem are engaged in software development, not retail.
My name is Denis and I am currently responsible for the backend of analytical systems at ISPsystem. And this is the story of how my colleague and I - responsible for data visualization - tried to look at our software products through the prism of this knowledge. Let's start, as usual, with history.
In the beginning there was a word, and the word was "Let's try?"
At that moment, I was working as a developer in the R&D department. It all started with the fact that here, on Habré, Danil read — a tool for analyzing user transitions in applications. I was somewhat skeptical about the idea of its application here. As examples, the developers of the library cited the analysis of applications where the target action was clearly defined - placing an order or another variation of how to pay the owner company. Our products are delivered on-premise. That is, the user first buys a license, and only then begins his journey in the application. Yes, we have demo versions. You can try the product in them so as not to take a pig in a poke.
But most of our products are focused on the hosting market. These are large clients, and the business development department advises them about the possibilities of the product. It also follows that at the time of purchase, our customers already know what tasks our software will help them solve. Their routes in the application should match those laid down in the CJM product, and UX solutions will help them not to stray. Spoiler: this is not always the case. Acquaintance with the library was postponed ... but not for long.
Everything changed with the release of our startup — — platforms for creating an online store from an Instagram account. In this application, the user was given a two-week period to use all the functionality for free. After that, it was necessary to decide whether to subscribe. And this fits perfectly into the concept of “route-target action”. It was decided: let's try!
First results or where to get ideas from
The development team and I connected the product to the event collection system literally in a day. I will say right away that ISPsystem uses its own system for collecting events about visiting pages, but nothing prevents you from using Yandex.Metrika for the same purposes, which allows you to upload raw data for free. Examples of using the library were studied, and after a week of data collection, we received a transition graph.
Transition graph. Basic functionality, other transitions removed for clarity
It turned out just like in the example: planar, understandable, beautiful. From this graph, we were able to identify the most frequent routes and crossings where people linger the longest. This allowed us to understand the following:
- Instead of a large CJM, which covers about a dozen entities, only two are actively used. We need to additionally direct users to the places we need with the help of UX solutions.
- Some of the pages that UX designers intended to be end-to-end are where people spend an unreasonable amount of time. You need to figure out what are the stop elements on a particular page and adjust it.
- After 10 transitions, 20% of people began to get tired and quit the session in the application. And this is taking into account the fact that we had as many as 5 onboarding pages in the application! It is necessary to identify pages on which users regularly abandon sessions, and shorten the path to them. Even better: detect any regular routes and allow for a quick transition from the source page to the destination page. Something in common with ABC analysis and abandoned cart analysis, don't you think?
And here we have revised our attitude to the applicability of this tool for on-premise products. It was decided to analyze an actively sold and used product - . It is much more complicated, there are an order of magnitude more entities. We anxiously waited for what the transition graph would turn out to be.
Of disappointment and inspiration
Disappointment #1
It was the end of the working day, the end of the month and the end of the year at the same time - December 27th. Data has been accumulated, requests have been written. There were seconds before everything was processed, and we could look at the result of our labors to find out where the next working year would start. R&D department, product manager, UX designers, team leader, developers gathered in front of the monitor to see how user paths look in their product, but… we saw this:
Transition graph built by the Retentioneering library
Inspiration #1
Strongly connected, dozens of entities, non-obvious scenarios. It was only clear that the new working year would begin not with analysis, but with the invention of a way to simplify the work with such a graph. But I had a feeling that everything is much simpler than it seems. And after fifteen minutes of studying the sources, Retentioneering managed to export the constructed graph to the dot format. This made it possible to upload the graph to another tool - Gephi. And already there expanse for the analysis of graphs: laying, filters, statistics - just do what you need to configure the necessary parameters in the interface. With this thought, we left for the New Year's weekend.
Disappointment #2
After going to work, it turned out that while everyone was resting, our customers were studying the product. Yes, so diligently that events appeared in the vault that did not exist before. This meant that the requests needed to be updated.
A little background to understand the sadness of this fact. We have both the events marked up by us (for example, clicking on some buttons) and the URLs of the pages that the user visited. In the case of Cartbee, the “one action, one page” model worked. But with VMmanager, the situation was already completely different: several modal windows could open on one page. In them, the user could solve various tasks. For example URL:
/host/item/24/ip(modal:modal/host/item/ip/create)
means that on the "IP Addresses" page, the user added an IP address. And there are two problems here:
- The URL contains some kind of path parameter - the ID of the virtual machine. It needs to be excluded.
- The URL contains the ID of the modal window. You need to "unpack" such URLs somehow.
Another problem was that those very events we marked up had parameters. For example, there were five different ways to get to the page with information about a virtual machine from the list. Accordingly, the event was sent one, but with a parameter that indicated which of the ways the user made the transition. There were many such events, and all the parameters are different. And we have all the data extraction logic in SQL dialect for Clickhouse. Requests for 150-200 lines began to seem like something familiar. Problems surrounded us.
Inspiration #2
One early morning, Danil, sadly scrolling the request for the second minute, suggested to me: “Let's write data processing pipelines?” We thought about it and decided what to do, then something like ETL. So that it filters immediately, and pulls up the necessary data from other sources. Thus, our first analytical service with a full-fledged backend was born. It implements five main stages of data processing:
- Unloading events from the storage of "raw" data and preparing them for processing.
- Refinement - "unpacking" those same identifiers of modal windows, event parameters and other details specifying the event.
- Enrichment (from the word "become rich") - supplementing events with data from third-party sources. At that time, this included only our billing system BILLmanager.
- Filtering is the process of screening out events that distort the results of the analysis (events from internal stands, emissions, etc.).
- Uploading the received events to the storage, which we called pure data.
Now it was possible to keep up-to-date by adding event processing rules or even groups of similar events. For example, since that moment we have never updated URL unpacking. Although, several new URL variations have been added during this time. They correspond to the rules already laid down in the service and are correctly processed.
Disappointment #3
Once we started the analysis, we realized why the graph was so connected. The fact is that almost every N-gram contained transitions that could not be implemented through the interface.
A little investigation began. I was embarrassed that there were no impracticable transitions within the same entity. This means that this is not a bug in the event collection system or our ETL service. There was a feeling that the user simultaneously works in several entities, without moving from one to another. How to achieve this? Using different tabs in the browser.
When analyzing Cartbee, we were saved by its specificity. The application was used from mobile devices, where it is simply inconvenient to work from several tabs. Here is a desktop and, while a task is being performed in one entity, it is reasonable to want to take this time with settings or monitoring the state in another. And in order not to lose progress, just open another tab.
Inspiration #3
Colleagues from fronted development taught the event collection system to distinguish between tabs. We could start the analysis. And we started. As expected, CJMs did not follow real paths: users spent a lot of time on catalog pages, leaving sessions and tabs in the most unexpected places. With the help of transition analysis, we were able to find problems in some Mozilla builds. In them, due to implementation features, navigation elements disappeared or half-empty pages were displayed, which should be available only to the administrator. The page opened, but the content from the backend did not come. The count of transitions made it possible to assess which features are actually used. Chains made it possible to understand how the user received this or that error. The data allowed testing based on user behavior. It was a success, the idea was not in vain.
Analytics automation
In one of the results demos, we showed how Gephi is used to analyze graphs. In this tool, navigation data can be displayed in a table. And the head of the UX department said one very important thought that influenced the development of the entire direction of behavior analytics in the company: “Let's do the same, but in Tableau and with filters, it will be more convenient.”
Then I thought: why not, Retentioneering stores all data in a pandas.DataFrame structure. And this is, by and large, a table. So another service appeared: Data Provider. He not only made a table out of the graph, but also calculated how popular the page and the functionality associated with it are, how it affects user retention, how long users stay on it, which pages users leave most often. And the use of visualization in Tableau has reduced the cost of studying the graph so much that the iteration time for analyzing behavior in the product has been reduced by almost half.
Danil will tell about how this visualization is applied and what conclusions can be drawn.
More tables for the god of tables!
In a simplified form, the task was formulated as follows: display the transition graph in Tableau, provide the ability to filter, make it as understandable and convenient as possible.
I didn’t really want to draw a directed graph in Tableau. And in case of success, the gain, in comparison with Gephi, seemed unobvious. Something much simpler and more affordable was needed. Table! After all, the graph can be easily represented as table rows, where each row is an edge of the “source-destination” type. Moreover, we have already carefully prepared such a table using Retentioneering and Data Provider tools. The matter remained small: display the table in Tableau and rummage through the report.
Speaking of how everyone loves tables
However, here we are faced with another problem. What to do with the data source? It was impossible to connect pandas.DataFrame, Tableau does not have such a connector. Raising a separate base to store the graph seemed too radical a decision with vague prospects. And the options for local unloading were not suitable due to the need for constant manual operations. We leafed through the list of available connectors, and our eyes fell on the item , which lonely huddled at the very bottom.
Tableau has a rich selection of connectors. Found one that solved our problem
What kind of animal? Several new open tabs in the browser - and it became clear that this connector allows you to receive data when accessing the URL. The backend for calculating the data itself was almost ready, it only remained to make friends with WDC. For several days, Denis studied the documentation and fought with Tableau mechanisms, and then threw me a link that I inserted into the connection window.
Form of connection to our WDC. Denis made his front and took care of safety
After a couple of minutes of waiting (the data is calculated dynamically when requested), a table appeared:
This is what a raw data array looks like in the Tableau interface
As promised, each row of such a table represented an edge of the graph, that is, a directed user transition. It also contained several additional features. For example, the number of unique users, the total number of transitions, and others.
It would be possible to display this table in the report as is, generously sprinkle filters and send the tool to the sea. Sounds logical. What can be done with the table? But this is not our way, because we are not just making a table, but a tool for analyzing and making product decisions.
As a rule, when analyzing data, a person wants to get answers to questions. Great. Let's start with them.
- What are the most frequent transitions?
- Where do specific pages go?
- How long, on average, spend on this page before leaving?
- How often does the transition from A to B?
- And on what pages does the session end?
Each of the reports or their combination should allow the user to independently find answers to these questions. The key strategy here is to provide a tool for doing it yourself. This is useful both to reduce the load on the analytics department and to reduce the time to make a decision - after all, you no longer need to go to Youtrack and start an analyst task, just open a report.
What did we get?
Where do they most often diverge from the dashboard?
A fragment of our report. After the dashboard, everyone went either to the list of VMs or to the list of nodes
Let's take a general table with transitions and filter by the source page. Most often, they go from the dashboard to the list of virtual machines. Moreover, the Regularity column suggests that this is a repetitive action.
Where do the list of clusters come from?
Filters in reports work in both directions: you can find out where you left, or you can find out where you went
It can be seen from the examples that even the presence of two simple filters and the ranking of rows by values allow you to quickly get information.
Let's ask something more difficult.
Where do users most often drop a session from?
VMmanager users often work in separate tabs
To do this, we need a report whose data is aggregated by referral sources. And the so-called breakepoints were taken as appointments - events that served as the end of the chain of transitions.
It is important to note here that this can be both the end of the session and the opening of a new tab. The example shows that the chain most often ends at a table with a list of virtual machines. At the same time, the characteristic behavior is a transition to another tab, which is consistent with the expected pattern.
We first of all checked the usefulness of these reports on ourselves when we conducted an analysis in this way. , another one of our products. With the advent of tables and filters, hypotheses were tested faster and eyes were less tired.
When developing reports, we did not forget about the visual design. When working with tables of this size, this is an important factor. For example, we used a calm range of colors, easy to perceive for numbers, color highlighting of lines in accordance with the numerical values of the characteristics. Such details enhance the user experience and increase the likelihood of a successful takeoff of the tool within the company.
The table turned out to be quite voluminous, but we hope it has not ceased to be readable.
Separately, it is worth mentioning the training of our internal clients: product managers and UX designers. For them, manuals were specially prepared with analysis examples and tips when working with filters. We have inserted links to manuals directly into the report pages.
We made the manual simply in the form of a presentation in Google Docs. Tableau tools allow you to display web pages right inside a book with reports.
instead of an epilogue
What's in the dry matter? We were able to get a tool for every day relatively quickly and cheaply. Yes, it's definitely not a replacement for the graph itself, the heat map of clicks, or the web viewer. But such reports significantly complement the listed tools, provide food for thought and new product and interface hypotheses.
This story was just the beginning for the development of analytics in ISPsystem. Over the past six months, seven more new services have appeared, including digital user portraits in the product and a service for creating bases for Look-alike targeting, but more about them in the next episodes.
Source: habr.com