Hello, Habr. This article is dedicated to the results of four years of developing a set of methods and tools for processing user movement trajectories in an application or website. The author of this development is , who leads the team of product creators and is also the author of this article. The product has been named Retentioneering and has now been converted into an open-source library hosted on GitHub, allowing anyone to make use of it. This may be of interest to those involved in product and marketing analytics, promotion, and product development. By the way, an article about one of the cases of working with Retentioneering has already been published on Habr. After reading this article, you will be able to write your own Retentioneering; it can be any standardized method for processing user trajectories within and beyond the application that allows you to see detailed behavioral features and extract insights for improving business metrics.
What is Retentioneering and why is it needed?
Initially, our goal was to translate Growth Hacking from the world of "digital wizardry" into the realm of numbers, analytics, and forecasting. As a consequence, product analytics was reduced to pure mathematics and programming for those who prefer numbers over fantastical stories, and formulas over clever buzzwords like "rebranding," "repositioning," etc., which sound nice but are of little practical help.
To solve these tasks, a framework was needed for analytics through graphs and trajectories, along with a library that simplifies typical routines of an analyst as a means of describing regular tasks in product analytics in a manner understandable to both humans and machines. The library allows for describing user behavior and linking it to the product's business metrics in such a formal and clear language as to simplify and automate routine tasks for developers and analysts and ease their communication with the business.
To address the specified tasks, we needed a framework for analytics through graphs and trajectories, as well as a library that simplifies typical routines for analysts. This would serve as a way to describe regular product analytics tasks in a manner understandable to both people and machines. The library allows for describing user behavior and linking it to the product's business metrics in such a formal and clear language that it simplifies and automates routine tasks for developers and analysts, thereby enhancing their communication with the business.
Retentioneering is a method and analytical software tools that can be adapted and integrated into any digital (and non-digital) product.
We started working on the product in 2015. Now it is a ready set of tools in Python and Pandas for data handling, machine learning models with a sklearn-like API, and interpretation tools for machine learning model results such as eli5 and shap.
All of this is wrapped Using the library is not difficult; virtually anyone who loves product analytics but has not coded before can apply our analytical methods to their data independently and without significant time expenditure.
Moreover, a programmer, app creator, or member of the development or testing team who has never engaged in analytics before can start playing with this code and observe usage patterns of their application without external help.
User trajectory as a fundamental element of analysis and methods for its processing.
The user trajectory is a sequence of the user's states at specific points in time. Moreover, events can come from different data sources, both online and offline. Events that happen to the user are part of their trajectory. Examples include:
• clicked a button
• saw an image
• landed on a screen
• received an email
• recommended a product to a friend
• filled out a form
• tapped the screen
• scrolled
• approached the checkout
• ordered a burrito
• ate the burrito
• got food poisoning from the eaten burrito
• entered the cafe through the back door
• entered through the front door
• closed the app
• received a push notification
• stared at the screen for too long
• paid for the order
• picked up the order
• was denied a loan
If you take the trajectory data of a group of users and study how transitions occur, you can trace how their behavior within the application is constructed. This can conveniently be done through a graph where states are nodes and transitions between states are edges:
"Trajectory" is a very convenient concept — it encompasses detailed information about all user actions, with the possibility of adding any additional data to describe these actions. This makes it a universal object. If you have beautiful and convenient tools that allow you to work with trajectories, you can find similarities and segment them.
At first, segmenting trajectories may seem very complicated. In a typical situation, it is — you need to use adjacency matrix comparison or sequence alignment. However, we found a simpler way — by studying a large number of trajectories and segmenting them through clustering.
It turns out that you can transform a trajectory into a point using continuous representations, for example, . After transformation, the trajectory becomes a point in space, where the axes represent the normalized frequency of various events and transitions within the trajectory. This concept from a vast space of a thousand or more dimensions (dimS=sum(event types)+sum(ngrams_2 types)) can be projected onto a plane using . TSNE is a transformation that reduces the dimensionality of the space to 2 axes while preserving relative distances between points as much as possible. Consequently, it becomes possible to analyze different trajectories on a flat map, a figurative projection map of trajectories, examining how the points of various trajectories relate to each other. It analyzes how close or distinct they were, whether they formed clusters or dispersed across the map, etc.:
The analytical tools of Retentioneering allow you to turn a complex array of data and trajectories into a representation that can be compared, and then to explore and interpret the results of this transformation.
When referring to standard methods of trajectory processing, three main tools implemented in Retentioneering are meant — graphs, step matrices, and trajectory projection maps.
Working with Google Analytics, Firebase, and similar analytics systems can be quite complex and not 100% effective. The problem lies in several user limitations, which result in analytics being restricted to mouse clicks and segment selections. Retentioneering allows for working with user paths rather than just funnels, as seen in Google Analytics, where the level of detail often reduces to the funnel, even if built for a specific segment.
Retentioneering and case studies
An example of using the developed tool can be demonstrated through a major niche service in Russia. This company has a mobile application for Android that is popular among clients. The annual revenue from the mobile application was about 7 million RUB, with seasonal fluctuations ranging from 60,000 to 130,000 RUB. The company also has an iOS app, and the average order value for user transactions on the 'Apple' app was higher than that of the customer using the Android app — 1,080 RUB compared to 1,300 RUB.
The company decided to increase the effectiveness of the Android app and conducted a thorough analysis. Several dozen hypotheses were formed about improving the app's performance. After using Retentioneering, it became clear that the issue was with the messages displayed to new users. They were receiving information about the brand, the company's advantages, and prices. However, it turned out that the messages needed to help users learn to navigate the app.
This was addressed, resulting in fewer app deletions, and the conversion rate for orders increased by 23%. Initially, 20 percent of incoming traffic was allocated for testing, but after a few days, analyzing the initial results and assessing the trend, the proportions were reversed, keeping 20 percent for the control group and assigning eighty percent to the test group. Another week later, the decision was made to sequentially test two more hypotheses. Overall, after seven weeks, the revenue from the Android app increased by one and a half times compared to the previous level.
How to work with Retentioneering?
The first steps are quite simple — we load the library using the command pip install retentioneering. The repository contains ready-made examples and case studies for some product analytics tasks. The set is constantly being expanded, but it's sufficient for a first acquaintance. Anyone can take the ready modules and immediately apply them to their tasks — this allows for quick and effective configuration of a more detailed analysis and optimization of user journeys. All this enables finding application usage patterns through understandable code and sharing this knowledge with colleagues.
Retentioneering is a tool that should be used throughout the entire lifecycle of an application, and here's why:
- Retentioneering is effective for tracking and continuously optimizing user journeys and improving business metrics. In e-commerce applications, new features are often added, the impact of which on the product cannot always be accurately predicted. In some cases, compatibility issues arise between new and old functions — for example, the new ones may 'cannibalize' existing ones. In this situation, continuous trajectory analysis is essential.
- A similar situation occurs when working with advertising channels: new traffic sources and ad creatives are constantly tested, requiring tracking of seasonality, trends, and the influence of other events, which leads to the emergence of new classes of problems. Here, too, constant monitoring and interpretation of user mechanics is needed.
- There are a number of factors that constantly affect the operation of an application. For example, new releases from developers: while solving a current problem, they inadvertently reintroduce an old one or create an entirely new issue. Over time, the number of new releases grows, and the process of bug tracking needs to be automated — including through user trajectory analysis.
Overall, Retentioneering is an effective tool. But there is no limit to perfection — it can and should be improved, developed, and built upon to create new great products. The more active the project's community is, the more forks will appear, leading to new interesting use cases.
Additional information about Retentioneering tools:
- Our website:
- and registration for participation
- Free
Source: habr.com
