Keeping a user in a mobile application is a whole science. Its basics in our article on VC.ru were described by the author of the course Maxim Godzi, Head of Machine Learning at App in the Air. Maxim talks about the tools developed by the company using the example of the analysis and optimization of a mobile application. This systematic approach to product improvement developed at App in the Air is called Retentioneering. You can also use these tools in your product: some of them are available in on GitHub.
App in the Air is an application with more than 3 million active users worldwide, with which you can track flights, get information about changes in departure / landing times, check-in and airport characteristics.
From funnel to trajectory
All development teams build an onboarding funnel (a process aimed at user acceptance of a product). This is the first step that helps you look at the whole system from above and find application problems. But as the product develops, you will feel the limitations of this approach. With a simple funnel, you cannot see non-obvious growth points for a product. The purpose of the funnel is to give you an overview of the stages of users in the application, to show you the metrics of the norm. But the funnel will prudently hide deviations from the norm in the direction of obvious problems or, on the contrary, special user activity.
We built our own funnel at App in the Air, but due to the specifics of the product, we ended up with an hourglass. Then we decided to expand the approach, and use the rich information that the application itself gives us.
When you build a funnel, you lose user onboarding trajectories. Trajectories consist of a sequence of user actions and the application itself (for example, sending a push notification).
Using timestamps, you can very easily restore the user's trajectory and make a graph out of it for each of them. Of course, there are a lot of graphs. Therefore, you need to group similar users. For example, you can arrange all users in the rows of a table and describe how often they use a certain function.
Based on such a table, we made a matrix and grouped users according to the frequency of using functions, that is, according to the nodes in the graph. This is usually the first step towards insights: for example, already at this stage you will see that some users do not use some of the functions at all. When we did a frequency analysis, we began to study which nodes of the graph are βlargestβ, that is, which pages users visit most often. Categories that are fundamentally different according to some criterion that is important to you immediately stand out. For example, here are two user clusters that we divided based on the subscription decision (there were 16 clusters in total).
How to use it
By looking at your users in this way, you can see what features you use to keep them or, for example, achieve a subscription. Naturally, the matrix will also show obvious things. For example, that those who bought a subscription were on the subscription screen. But besides this, you can find patterns that you would never have known about otherwise.
So we accidentally found a group of users who add a flight, actively track it during the day and then disappear for a long time until they fly somewhere again. If we analyzed their behavior using conventional tools, we would think that they were simply not satisfied with the functionality of the application: how else to explain that they used it for one day and never returned. But with the help of graphs, we saw that they are very active, it's just that all their activity fits into one day.
Now our main task is to push such a user to connect the loyalty program of his air carrier while he uses our statistics. In this case, we'll import all the flights he buys and try to push him to subscribe as soon as he buys a new ticket. To solve this problem, we also began to cooperate with Aviasales, Svyaznoy.Travel and other applications. When their user buys a ticket, the application prompts him to add the flight to App in the Air, and we immediately see it.
Thanks to the graph, we saw that 5% of the people who go to the subscription screen abandon it. We began to analyze such cases, and saw that there is a user who enters the first page, initiates the connection of his Google account, and immediately cancels it, again gets to the first page, and so four times. At first we thought: "Something is clearly wrong with this user." And then we realized that, most likely, there is a bug in the application. On the funnel, this would be interpreted as follows: the user did not like the permission set that the application requests, and he left.
For another group, 5% of users got lost on the screen where the app prompts them to select one of all the calendar apps on their smartphone. Users chose different calendars over and over again, and then simply exited the application. It turned out that there was a UX problem here: after the person selected the calendar, he had to click Done in the upper right corner. It's just that not all users saw it.
First screen of App in the Air
In our graph, we saw that about 30% of users do not go beyond the first screen: this is due to the fact that we aggressively push the user to subscribe. On the first screen, the app prompts you to register with Google or Triplt, and there is no information that you can skip registration. Of those who leave the first screen, 16% of users click "More" and come back again. We found out that they are looking for a way to register internally in the app and we will release it in the next update. In addition, 2/3 of those who immediately leave do not press anything at all. To find out what happens to them, we built a heatmap - a heat map. It turned out that customers click on a list of app features that are not clickable links.
Catching a micromoment
You can often see people treading paths next to the paved road. Retentioneering is an attempt to find these paths and change the roads where possible.
Of course, itβs bad that we learn from real users, but at least we began to automatically track patterns that indicate a user problem in the application. Now the product manager receives email notifications if there are a large number of "loops" - when the user returns to one screen again and again.
Let's take a look at what patterns in user trajectories are generally interesting to look for in order to analyze the problems and growth points of the application:
- Loops and cycles. The loops mentioned above are when one event repeats in the user's trajectory, such as calendar-calendar-calendar-calendar. A loop with a lot of repetitions is a clear indication of a problem with the interface or insufficient markup with events. A cycle is also a closed trajectory, but unlike a loop, it includes more than one event, for example: viewing flight history - adding a flight - viewing flight history.
- Flowstoppers - when the user, due to some kind of obstacle, cannot continue his desired movement through the application, for example, a screen with an interface that is not obvious to the client. Such events slow down and shift the trajectory of users.
- Bifurcation points are milestone events after which the trajectories of clients of different types are separated. In particular, these are screens that do not contain a direct transition or call-to-action to the target action, effectively pushing some users to it. For example, some screen that is not directly related to the purchase of content in the application, but on which customers tend to buy or not buy content, will behave differently. Bifurcation points can be points of influence on the actions of your users with a plus sign - to influence the decision to purchase or the right click, or minus - they can determine that after a few steps the user will leave the application.
- Aborted conversion points are potential bifurcation points. You can think of them as screens that could lead to a target action but don't. It can also be a moment in time when the user has a need, but we do not satisfy it, because we simply do not know about it. Analysis of the trajectory should allow this need to be identified.
- Distraction point - screens / pop-ups that do not bring value to the user, do not affect the conversion, and at the same time can βblurβ the trajectories, distracting the user from the target actions.
- Blind spots are hidden application points, screens and features that are very difficult for the user to reach.
- Drains - points where traffic leaks
In general, the mathematical approach allowed us to understand that the client uses the application in a completely different way than product managers usually think, trying to plan some kind of standard usage scenario for the user. Sitting in the office and attending the coolest product conferences, it is still very difficult to imagine all the variety of real field conditions in which the user will solve his problems using the application.
This sounds like a cool joke. A tester walks into a bar and orders: a mug of beer, 2 mugs of beer, 0 mugs of beer, 999999999 mugs of beer, a lizard in a glass, -1 mug of beer, qwertyuip mugs of beer. The first real customer walks into the bar and asks where the toilet is. The bar bursts into flames, everyone dies.
Product analysts, deeply immersed in this problem, began to introduce the concept of a micromoment. The modern user needs an instant solution to their problem. Google started talking about this a few years ago: the company called such user actions micromoments. The user is distracted, accidentally closes the application, does not understand what is required of him, logs in again a day later, forgets again, and then follows the link sent to him by a friend in the messenger. And all these sessions can last no more than 20 seconds.
So we started trying to set up the service desk so that employees could understand what the problem was almost in real time. By the time a person has arrived at the support contact page and started writing their question, we can determine the root of the problem by knowing its trajectory - the last 100 events. Previously, we automated the categorization of all support tickets using ML analysis of support request texts. Despite the success of categorization, when 87% of all calls are correctly distributed into one of 13 categories, it is the work with trajectories that is able to automatically find the most appropriate solution for the user's situation.
We cannot release updates quickly, but we are able to notice the problem and if the user follows the scenario that we have already seen, send him a push notification.
We see that the task of optimizing the application requires a rich toolkit for the study of user trajectories. Further, knowing all the paths that users take, you can lay the right paths, and with the help of customized content, push notifications and adaptive UI elements βby the handβ, lead the user to targeted actions that best suit their needs and bring money, data and other value for your business.
What to take into account
- To study user conversion only on the example of funnels is to lose the rich information that the application itself gives us.
- Retentioneering - analysis of user trajectories on graphs helps to see what features you use to retain users or, for example, stimulate them to subscribe.
- Retentioneering tools help to automatically track patterns in real time that indicate a user problem in the application, find and close bugs where it was hard to notice them.
- They help to find non-obvious patterns of user behavior.
- Retentioneering tools make it possible to build automated ML tools for predicting key user events and metrics: user loss, LTV and many other metrics that are easily defined on a graph.
We are building a community around Retentioneering for the free exchange of ideas. You can think of the toolkit that we are developing as a language in which analysts and products from different mobile and web applications can share insights, best practices and methods. You can learn how to use these tools on the course Binary District.
Source: habr.com