I have always been interested in how Habr is organized from the inside, how workflow is built, how communications are built, what standards are applied and how code is written here. Fortunately, I had such an opportunity, because I recently became part of the habra team. Using the example of a small refactoring of the mobile version, I will try to answer the question: what is it like to work here on the front. In the program: Node, Vue, Vuex and SSR with sauce from notes about personal experience in Habré.
The first thing you need to know about the development team is that we are few. Not enough - these are three fronts, two backs and the technical lead of all Habr - Baxley. Of course, there is also a tester, a designer, three Vadims, a miracle broom, a marketer and other Bumburums. But there are only six direct contributors to Habr's sources. This is quite rare - a project with a multi-million audience, which looks like a giant enterprise from the outside, in fact looks more like a cozy startup with the most flat organizational structure.
Like many other IT companies, Habr professes the ideas of Agile, the practice of CI, and that's it. But according to my feelings, Habr as a product develops rather in waves than continuously. So for several sprints in a row we diligently code something, design and redesign, break something and fix it, resolve tickets and start new ones, step on a rake and shoot ourselves in the foot to finally release a feature in production. And then there is some calm, a period of redevelopment, a time to do what is in the "important-non-urgent" quadrant.
Just such an “off-season” sprint will be discussed below. This time, the refactoring of the mobile version of Habr got into it. In general, the company has high hopes for it, and in the future it should replace the entire zoo of Habr's incarnations and become a universal cross-platform solution. Someday there will be adaptive layout, and PWA, and offline mode, and user customization, and many other interesting things.
We set the task
Once at an ordinary stand-up, one of the fronts spoke about problems in the architecture of the comments component of the mobile version. With this suggestion, we organized a micro-meeting in the format of group psychotherapy. Everyone in turn said where it hurts, everything was recorded on paper, they sympathized, they understood, except that no one clapped. The output was a list of 20 problems, which made it clear that the mobile Habr still has a long and thorny path to success.
I was primarily concerned about resource efficiency and what is called a smooth interface. Every day on the home-work-home route, I saw my old phone frantically trying to display 20 headings in the feed. It looked something like this:
Mobile Habr interface before refactoring
What's going on here? In short, the server gave the HTML page to everyone the same, regardless of whether the user is logged in or not. Then the client JS is loaded and re-requests the necessary data, but adjusted for authorization. That is, in fact, we did the same job twice. The interface flickered, and the user downloaded a good hundred extra kilobytes. In detail, everything looked even more creepy.
Old SSR-CSR scheme. Authorization is possible only at stages C3 and C4, when Node JS is not busy generating HTML and can proxy API requests.
Our architecture of that time was very accurately described by one of Habr's users:
The mobile version is crap. I speak as it is. Terrible combination of SSR with CSR.
We had to admit it, no matter how sad it was.
I figured out the options, put myself a ticket in Jira with a description at the level of “now it’s bad, do it right” and decomposed the task in broad strokes:
- reuse data
- minimize the number of redraws,
- exclude duplicate requests,
- make the boot process more obvious.
Reusing data
In theory, server-side rendering is designed to solve two problems: not to suffer from the limitations of search engines in terms of and improve the metric (inevitably worsening ). In the classic scenario, which is definitively year (still on Backbone.js), SSR is the same isomorphic JS application running in a Node. The server simply returns the generated layout as a response to the request. Then rehydration takes place on the client side, and then everything works without page reloads. For Habr, as well as for many other resources with text content, server rendering is a critical element in building friendly relations with search engines.
Despite the fact that more than six years have passed since the emergence of the technology, and during this time a lot of water has flowed under the front-end world, for many developers this idea is still shrouded in a veil of secrecy. We did not stand aside, and rolled out a Vue application with SSR support, missing one small detail: we did not pass the initial state to the client.
Why? There is no exact answer to this question. Either they didn’t want to increase the size of the response from the server, or because of a bunch of other architectural problems, or they simply didn’t take off. One way or another, throwing state and reusing everything that the server did seems to be quite expedient and useful. The task is actually trivial - to the execution context, and Vue automatically adds it to the generated layout as a global variable: window.__INITIAL_STATE__.
One of the problems that has arisen is the inability to convert cyclic structures to JSON (); was solved by simply replacing such structures with their flat counterparts.
In addition, when dealing with UGC content, you should remember that the data should be converted to HTML entities in order not to break the HTML. For these purposes, we use .
Minimizing redraws
As you can see from the diagram above, in our case, one Node JS instance performs two functions: SSR and “proxy” in the API, where the user is authorized. This circumstance makes it impossible to authorize at the time of execution of the JS code on the server, since the node is single-threaded, and the SSR function is synchronous. That is, the server simply cannot send requests to itself while the call stack is busy with something. It turned out that we threw the state, but the interface did not stop twitching, since the data on the client should have been updated taking into account the user session. It was necessary to teach our application to put the correct data in the initial state, taking into account the user's login.
There were only two solutions to the problem:
- attach authorization data to interserver requests;
- split the layers of Node JS into two separate instances.
The first solution required the use of global variables on the server, while the second extended the timeframe for the task to be completed by at least a month.
How to make a choice? Habr often moves along the path of least resistance. Informally, there is a certain general desire to reduce the cycle from idea to prototype to a minimum. The model of attitude to the product is somewhat reminiscent of the postulates of booking.com, with the only difference being that Habr takes user feedback much more seriously and trusts you as a developer to make such decisions.
Following this logic and my own desire to quickly solve the problem, I chose global variables. And, as often happens, sooner or later you have to pay for them. We paid almost immediately: we worked on the weekend, sorted out the consequences, wrote and began to divide the server into two parts. The error was very stupid, and the bug with her participation was not easily reproduced. And yes, it’s a shame for this, but anyway, stumbling and groaning, my PoC with global variables still went into production and is working quite successfully in anticipation of moving to a new “two-day” architecture. This was an important step, because formally the goal was achieved - SSR learned to give a completely ready-to-use page, and the UI became much more relaxed.
Mobile Habr interface after the first stage of refactoring
Ultimately, the SSR-CSR architecture of the mobile version leads to this picture:
"Two-node" scheme SSR-CSR. Node JS API is always ready for asynchronous I / O and is not blocked by the SSR function, since the latter is in a separate instance. Request chain #3 is not needed.
Eliminate duplicate requests
After the manipulations, the initial rendering of the page stopped provoking epilepsy. But the further use of Habr in SPA mode was still puzzling.
Since the user flow is based on transitions of the form list of articles → article → comments and vice versa, it was important to optimize the resource consumption of this chain in the first place.
Return to the post feed provokes a new request for data
Didn't have to dig deep. The screencast above shows that the application re-requests the list of articles when swiping back, and during the request we do not see the articles, which means that the previous data disappears somewhere. It looks like the article list component uses the local state and loses it on destroy. In fact, the application used a global state, but the Vuex architecture was built “on the forehead”: modules are attached to pages, which in turn are attached to routes. Moreover, all modules are “one-time” - each next visit to the page rewrote the entire module:
ArticlesList: [
{ Article1 },
...
],
PageArticle: { ArticleFull1 },
In total, we had a module ArticlesList, which contains objects of type Paper and module PageArticle, which was an extended version of the object Paper, kind of ArticleFull. By and large, this implementation does not carry anything terrible in itself - it is very simple, one might even say naively, but it is extremely understandable. If you cut out the reset of the module with each change of route, then you can even live with it. However, the transition between the feeds of articles, for example /feed → /all, is guaranteed to throw out everything related to the personal feed, since we only have one ArticlesList, in which you want to put the new data. This again leads us to duplicate requests.
Having gathered together everything that I managed to unearth on the topic, I formulated a new state structure and presented it to my colleagues. The discussions were lengthy, but in the end, the pros outweighed the doubts, and I set about implementing it.
The solution logic is best revealed in two steps. First we try to decouple the Vuex module from the pages and bind directly to the routes. Yes, there will be a little more data in the store, getters will become a little more complicated, but we will not load articles twice. For the mobile version, this is perhaps the strongest argument. It will turn out something like this:
ArticlesList: {
ROUTE_FEED: [
{ Article1 },
...
],
ROUTE_ALL: [
{ Article2 },
...
],
}
But what if lists of articles can overlap between multiple routes and what if we want to reuse object data Paper to render the post page, turning it into ArticleFull? In this case, it would be more logical to use the following structure:
ArticlesIds: {
ROUTE_FEED: [ '1', ... ],
ROUTE_ALL: [ '1', '2', ... ],
},
ArticlesList: {
'1': { Article1 },
'2': { Article2 },
...
}
ArticlesList here - it's just a kind of repository of articles. All articles that were downloaded during the user's session. We treat them as carefully as possible, because this is traffic that may have been downloaded through pain somewhere in the subway between stations, and we absolutely do not want to cause the user this pain again by forcing him to download data that he has already downloaded. An object ArticlesIds is just an array of IDs (like “links”) to objects Paper. This structure allows you not to duplicate data common to routes and reuse the object Paper when rendering a post page by merging extended data into it.
The output of the list of articles has also become more transparent: the iterator component iterates through the array with the article IDs and renders the article teaser component, passing the Id as a prop, and the child component, in turn, gets the necessary data from ArticlesList. When going to the publication page, we get the already existing date from ArticlesList, make a request to get the missing data and simply add it to the existing object.
Why is this approach better? As I wrote above, this approach is more careful with respect to the downloaded data and allows you to reuse them. But besides this, it opens the way to some new features that fit perfectly into such an architecture. For example, polling and uploading articles to the feed as they appear. We can just put the fresh posts in the "storage" ArticlesList, save a separate list of new IDs in ArticlesIds and notify the user about it. When we click on the "Show New Posts" button, we simply insert the new Ids at the beginning of the array of the current list of articles and everything will work almost magically.
Making loading more enjoyable
The icing on the cake of the refactoring was the concept of skeletons, which makes the process of loading content on a slow Internet a little less disgusting. There were no discussions on this subject, the path from idea to prototype took literally two hours. The design almost drew itself, and we taught our components to render simple, barely flickering divs while waiting for data. Subjectively, this approach to loading really reduces the amount of stress hormones in the user's body. The skeleton looks like this:
habraloading
Reflecting
I have been working in Habré for half a year and my friends still ask: well, how are you doing there? Okay, comfortable, yes. But there is something that distinguishes this work from others. I worked in teams that were absolutely indifferent to their product, did not know and did not understand who their users were. But here everything is different. Here you feel responsible for what you do. In the process of developing a feature, you partially become its owner, take part in all product meetings related to your functionality, make suggestions and make decisions yourself. Making a product that you use every day yourself is very cool, and writing code for people who probably understand it better than you is just an incredible feeling (no sarcasm).
After the release of all these changes, we received positive feedback, which was very, very nice. It's inspiring. Thank you! Write more.
Let me remind you that after the global variables, we decided to change the architecture and separate the proxy layer into a separate instance. The "two-day" architecture has already reached the release in the form of public beta testing. Now anyone can switch to it and help us make mobile Habr better. That's all for today. I will be happy to answer all your questions in the comments.
Source: habr.com