Almost fired. How I built the Yandex analytics department

Almost fired. How I built the Yandex analytics department My name is Alexey Dolotov, I haven't written to Habr for 10 years. Part of the reason is that when I was 22, I started building the Yandex analytics department, then led it for seven years, and now I came up with and build the Yandex.Talents service. The profession of an analyst provides a lot of opportunities. The main thing is to start right - for example, in School of managers Currently recruiting for analytics.

I decided to tell you how my career has developed and give some advice to those who want to "ignite" in this profession. I hope my unique experience will be useful to someone.

The only semester of the university and the beginning of a career

By the time I entered the university, I was good at programming, I even wrote my own shareware product (a word from the past). It was a laserdisc cataloger. Winchesters were still small, not everything fit on them, so people often used CDs and DVDs. The cataloger read the file system of the disk, indexed it and collected meta-information from the files, wrote it all into a database and allowed it to be searched. On the first day, 50 thousand Chinese downloaded the product, on the second day a crack appeared on Altavista. And I thought I made a cool defense.

I entered ITMO University in St. Petersburg, but after one semester I decided that I already know how to program, I learn faster in the process of work, and therefore I left for Norway to freelance. When I returned, I organized a web studio for two with a partner. He was rather responsible for business and documents, I was responsible for everything else, including the technical part. At different times, up to 10 people worked for us.

Yandex in those years held so-called client seminars, one of which I allegedly penetrated as a journalist. Andrey Sebrant, Zhenya Lomize, Lena Kolmanovskaya performed there. After listening to them speak, I was impressed by their out-of-the-box thinking. The best way to approach someone in terms of professionalism is to start working with them. Therefore, at that moment - I was 19 or 20 years old - I rethought my whole life, I decided to quit a not very successful web studio and move from St. Petersburg to Moscow to get into Yandex.

I did not manage to do this immediately after the move. The department, in which for some reason I stubbornly tried to get a job, knew that I had sneaked into the seminar mentioned above and even tried to get a certificate on the Yandex.Direct course I had completed. By the way, this certificate could not be issued to me for a long time. No one expected that anyone other than the main audience of the seminar would take the course. This story seemed strange to my future colleagues, and Yandex did not take me then.

But in Mail.Ru they took it quickly, five interviews in two days. It was by the way - after the move, I was already running out of money. I was responsible for all search services, including GoGo and go.mail.ru. But after a year and a half, I still moved to Yandex as a manager of sorcerers (search items that answer the user's question directly on the search results page). It was the end of 2008, about 400 people worked at Mail.Ru, about 1500 at Yandex.

Yandex

I must admit, Yandex did not succeed at first. Four months later, I was offered to look for other options in the company. In fact, they were fired. I had some time to search, but if I did not find anything, I would have to leave. Until then, I hadn't worked for a really big company with a complex project management structure. Not oriented, not enough experience.

I stayed, got a job as an analyst of communication services: Fotok, Ya.ru, but most importantly, Mail. And here a combination of managerial skills (like people, agree), product skills (understand where the benefits are, what users want) and technical skills (apply programming experience, process data on their own) came in handy for me.

We were the first in the company to start building cohorts - to study the dependence of the outflow of users on the month in which they registered. First, we fairly accurately predicted the volume of the audience using the resulting model. Secondly, and more importantly, it could predict how various changes would affect the main indicators of the service. Previously, Yandex did not do this.

Somehow Andrey Sebrant came to me and said - you are doing well, now we need the same on the scale of the entire Yandex. "Make a section." I replied: "Good."

Department

Andrei helped me a lot, including sometimes saying - you're an adult guy, figure it out. No typos, that's a help too. I really needed more independence, and I started doing everything myself. When a question arose for the management, I tried to think first: what would the leader answer me to this question? This approach helped to develop faster. Sometimes, because of the great responsibility, it was simply scary. A turning point has come: from a person who solves problems, I became responsible for the development of a large piece of processes. The number of services and the services themselves grew, they needed analysts. I was actively involved in two things: hiring and mentoring.

People often came to me with questions to which I did not know the answers. Thus, I learned to solve almost any problem with some accuracy, based on a very limited amount of data. It's like "What? Where? When? ”, Only there it is necessary to give the correct answer, and here there may not be a correct answer at all, but it is enough to understand in which direction to dig. I began to struggle with many cognitive biases (perhaps the most popular among researchers and analysts is confirmation bias, the tendency to confirm my point of view), I developed "invariant", "quantum" thinking. It works like this: you hear the condition of the problem and immediately imagine all possible and impossible solutions, automatically “solving” these branches and understanding what minimum hypotheses need to be tested in order to “solve” as many of the most probable branches as possible.

I also taught the guys what I did not know myself. I learned the first basics of statistics in the interviews that I conducted. Then he began to teach leadership, although he himself had just become a leader. There seems to be no greater incentive to understand something properly than to explain it to someone else.

partisans

I began to help analysts grow: I told everyone that I would work with him, and he should work independently with the service team. At the same time, I asked uncomfortable questions. An analyst comes to me and talks about the tasks that he is currently doing. Further dialogue:

Why are you doing such tasks?
Because they asked me to.
- And what are the most important tasks for the team itself now?
- I do not know.
- Let's do not what was asked, but what the service requires.

Next dialogue:

- They do something.
- Why don't they do it? What did they not take into account, what did they forget to think about?

I taught the guys not to take on tasks until they understand what really “hurts” the customer. It is important to “rehearse” the scenario of how the analytics result will be used together with the customer. It often turned out that the customer did not need what he initially asked for. Understanding this is the responsibility of the analyst.

This is the philosophy of “good guerrilla” or “guerrilla product management”. Yes, you are just an analyst. But you have the opportunity to influence the course of the entire service - for example, through the correct formulation of metrics. Formulating metrics and goals based on them is perhaps the main tool of influence for an analyst. A clear and transparent goal, decomposed into metrics, each of which is clear how to improve, is the best way to direct the team on the right course and help them stay on track. I promoted the idea that all my guys should interact cross-service and thereby form “hydrogen bonds” inside Yandex, which tended to fall apart at other seams.

Search share

In 2011, we investigated the reasons for the change in the search share of Yandex - it was difficult to prove the influence of each specific factor, and there were many of them. One Friday, I showed Arkady Volozh a chart that I had been unable to draw up for a long time and finally did. Then I came up with a "factor freeze method" that allowed me to highlight the influence of browsers with a pre-installed alternative search. It read well that the share is changing precisely because of them. Such a conclusion at that time did not seem obvious: people still did not use such browsers very often. And yet it turned out that the preset search has a strong influence on the situation.

In those days, the phase of my active communication with Volozh began: I began to devote more time to the search share. The concept itself appeared - share analytics or “fuckup” (drastic changes in the share were often caused by someone's fakap). It was then that Seryozha Linev appeared in the team, one of the key Yandex analysts in the future. Together with Lyosha Tikhonov, another excellent analyst and author of Autopoet, we helped Serezha grow and create around him invaluable expertise in detecting and analyzing complex anomalies. Now, if any incident affecting the share occurs, the administrator on duty immediately learns about it with all the details. It is no longer required, as then, to gather a dozen analysts and spend several days investigating the causes. We can say that now in this regard we have the era of spaceships, and then we dragged carts.

Arcadia has always been very interested in the share. He began to call and write to me frequently when anomalies appeared on the instruments of the search share - even if I had no relation to the causes of these anomalies. Maybe he kept calling me specifically because it helped. And I just knew who to call next.

By the way, Yandex has a mailing list for non-working questions, and when I asked someone to lend me a badminton racket, it was Arkady who responded first.

Ilya

Perhaps it is appropriate here to tell how I - albeit not for long - worked with Ilya Segalovich. Chronologically, this should have been told earlier: oddly enough, I worked with him while still at Mail.Ru.

The fact is that the search for go.mail.ru at that moment worked on the Yandex engine (only GoGo, another Mail.Ru project, had its own engine). Therefore, as a manager of search services, I was given contacts of several Yandexoids. For technical questions, I called either Tolya Orlov or Ilya Segalovich. To my shame, I had no idea at the time who these people were. During non-working hours, it was easier to get through to Ilya's work phone, but in the afternoon it was the other way around. I was surprised why he is so rarely in the workplace, I thought - what kind of developer is this? But when he answered, he very politely and meaningfully helped me in the shortest possible time. That's why I called him in the first place.

Later I found out who Ilya was, I even played badminton with him as part of a large group of colleagues. Getting settled in Yandex, I tried to remember what I managed to say to him. Ilya really, by all external signs, was an ordinary good person without any star disease.

There was a case, we ran into Ilya in the elevator. Ilya, wildly excited, arranges an elevator pitch for me, showing the screen of his phone: “This is the future!” During the time in the elevator it is impossible to figure out what exactly he means. But you notice how much a person burns, and you don’t understand whether this is madness or genius. Probably both.

There are people whose ideas live in me and make me better. Ilya is one of them.

Talents

Many of the current analytical departments in Yandex are now headed by my guys. There were several reasons why I decided to switch to something else after seven years in charge of the department.

Firstly, Yandex has become a group of companies, and the need for centralized analytics has disappeared. Secondly, with such a large department, there was too much administrative work. And thirdly, I wanted to make decisions and take full responsibility for them. I wanted to come home one day and say to my wife, "That's what I did."

Therefore, I created the Yandex.Talents service. We're trying to reimagine job hunting and hiring. We are only taking the first steps now, but I see great potential in us. The classic idea of ​​a job board in an era when machine learning is everywhere and drones are driving the streets seems outdated. It's time to start using smart algorithms to help both job seekers and employers.

I used to explain all the time to people in services how to do their job, considering that these arguments are based on analytics and my expert opinion. But the work on Yandex.Talents has shown that I am very often wrong. The truth is born between people - a simple statement, which, however, must be felt. In addition, the creation of a startup required a lot of immersion in the business, and now I believe that first of all, a product analyst should study the financial model of his product. If you don't understand what your key business metrics are, how can you help your team achieve them?

What a Great Analyst Needs

There are many things that an analyst has to be able to do, but there are two main skills that allow you to really “ignite”.

First of all, a phenomenal ability to deal with cognitive distortions is required. I recommend reading the article "List of Cognitive Distortions" on Wikipedia, a fascinating and useful reading. All the time you catch yourself thinking how much this list is about us.

And second, no authority can be recognized. Analytics is about arguing. You first prove to yourself that you yourself are wrong in your conclusions, and then you learn to prove that someone else is wrong. One day in August 2011, the Yandex portal worked intermittently for some time. It was Friday, and the next Monday there was a Khural led by me. Arkady came and swore for a long time. Then I took the floor: "Arkady, now I will start the Khural, perhaps." He says no, there will be no Khural, let everyone go to work. I replied that I would not let the company work all week in such a mood. He immediately agreed. And we held a khural.

These qualities will come in handy in other areas, especially if you are a leader.

Source: habr.com

Add a comment