Mail.ru Group tech issue 2019

Mail.ru Group tech issue 2019

At the end of May, our graduates from the Technopark (Bauman Moscow State Technical University), Technotrack (MIPT), Technosphere (Lomonosov Moscow State University) and Technopolis (Peter the Great St. Petersburg Polytechnic University) defended their diploma projects. Three months were allocated for work, and the guys invested in their offspring the knowledge and skills gained over two years of study.

In total, 13 projects were defended, solving various problems in different industries. For example:

  • cloud storage with cryptographic file encryption;
  • platform for creating interactive video (with different endings);
  • smart board for playing real chess over the network;
  • architecture for intellectual search of medical articles;
  • Software for teaching children of primary school age the basics of algorithmization.

As well as projects from business units:

  • CRM system for TamTam messenger;
  • web service for searching thematic photos on the map for Odnoklassniki;
  • address geocoding service for MAPS.ME.

Today we will tell you more about five projects of our graduates.

Intelligent search for medical articles

Mail.ru Group tech issue 2019

There are many areas in the scientific field, in each of which research is being conducted, a huge number of articles are published in a variety of journals. These are information technologies, physics, mathematics, biology, medicine and many others.

Authors project decided to focus on the medical field. Almost all articles on medical topics are collected on the PubMed portal. The portal provides its own search. However, its capabilities are very limited. Therefore, the guys improved the search system, added support for long queries and the ability to refine queries using topic modeling.

Mail.ru Group tech issue 2019
The search results contain a ranked list of documents with the definition of their topics, and the words and terms related to these topics are highlighted thanks to probabilistic topic modeling. The user can click on the highlighted terms to narrow down the search query.

Mail.ru Group tech issue 2019
To make searching through the huge PubMed database fast, the authors wrote their own search engine that can be easily integrated into any infrastructure.

The search is carried out in three stages:

  1. Candidate documents are selected using a reverse index.
  2. Candidates are ranked according to the BM25F algorithm, which takes into account various fields in documents when searching. Thus, the words in the title carry more weight than the words in the abstract.
  3. A caching system is also used to speed up the processing of frequent requests.

Mail.ru Group tech issue 2019

Microservice architecture:

Mail.ru Group tech issue 2019
Basically, structured text data is transferred between services. For high transfer rates, GRPC is used - a framework for communicating modules in a microservice architecture. Data is also serialized using the Protobuf message exchange format.

What components does the system include:

  • Server for processing incoming user requests on Node.js.
  • Load balancing requests with nginx proxy.
  • The Flask server implements a REST API and receives requests forwarded from Node.js.
  • All raw and processed data, as well as query information, is stored in MongoDB.
  • All requests for a relevant document theming output go to RabbitMQ.

Search result example:

Mail.ru Group tech issue 2019

What is planned to be done in the future:

  • Recommendations when compiling reviews on a given topic (identifying important topics in a document and searching through subsets of documents).
  • Search in PDF-files.
  • Semantic segmentation of texts.
  • Track topics and trends over time.

Project team: Fedor Petryaikin, Vladislav Dorozhinsky, Maxim Nakhodnov, Maxim Filin

block log

Mail.ru Group tech issue 2019

Today, when teaching programming and computer science to children of primary school age (grades 5-7), there are problems with mastering the material. In addition, if students want to complete assignments at home, they have to install additional software on their computers. Teachers have to check a large number of the same type of problem solutions, and in the case of remote learning, they also have to develop a methodology for receiving assignments from students.

The authors of the Block Log project came to the conclusion that when teaching primary school children the basics of algorithmization, the emphasis should not be on memorizing programming language commands, but on building algorithm schemes. This will allow students to spend time and effort on designing an algorithm, and not on stuffing cumbersome syntactic constructions.

Platform block log allows you to:

  1. Create and edit flowcharts.
  2. Run the created flowcharts and see the result of their work (output data).
  3. Save and load created projects.
  4. Draw bitmap images (image generation based on an algorithm created by the child).
  5. Get information about the complexity of the created algorithm (based on the number of operations performed in the algorithm).

The division of roles into teachers and students is supposed. Any user receives the status of a student, to obtain the status of a teacher, you must contact the system administrator. The teacher can not only enter the description and conditions of the problems, but also create automated tests that will be automatically launched when the student sends the solution to the problem to the system.

Block Log Browser Editor:

Mail.ru Group tech issue 2019

Having solved the problem, the student can download the solution and see the results:

Mail.ru Group tech issue 2019

The platform consists of a Vue.js front-end application and a Ruby on Rails back-end application. PostgreSQL is used as the database. To simplify deployment, all system components are packaged in Docker containers and assembled using Docker Compose. The desktop version of Block Log is based on the Electron framework. Webpack was used to build the JavaScript code.

Project team: Alexander Barulev, Maxim Kolotovkin, Kirill Kucherov.

CRM system for TamTam messenger

Mail.ru Group tech issue 2019

CRM is a tool for convenient business interaction with TamTam users. The following functions have been implemented:

  • bot constructor that allows you to create bots without programming skills. In a few minutes, you can get a fully working bot that can not only show some information to users, but also collect data, incl. files that the administrator can view later.
  • RSS. You can easily connect RSS to any feed.
  • Delayed posting. Allows you to send and delete messages at a predetermined time.

The team also took part in testing the Bot API by creating several self-written bots, such as a bot for the 2019 Ice Hockey World Championship, a bot for registration/authorization in our service, and a bot for CI/CD.

Solution Infrastructure:

  • The management server contains a monitoring system for each server and each Docker container on it in order to quickly and conveniently detect a problem and solve it, view various metrics and usage statistics. There is also a remote configuration management system for our application.
  • The staging server contains the latest version of our application, available for general testing by the development team.
  • The management and staging servers are only accessible via VPN to developers, while the production server contains the release version of the application. It is isolated from the hands of developers and is available only to the end user.
  • CI/CD system implemented using Github and Travis, notification using a custom bot in TamTam.

Mail.ru Group tech issue 2019

The application architecture is a modular solution. The application, database, configuration manager and monitoring run in separate Docker containers, which allows you to abstract from the launch environment, change or restart a separate container. Creating a network topology and managing containers is done using Docker Compose.

Mail.ru Group tech issue 2019

Project team: Alexey Antufiev, Yegor Gorbatov, Alexey Kotelevsky.

ForkMe

Mail.ru Group tech issue 2019

The ForkMe project is an interactive video viewing platform where you can design your video and show it to your friends. Why do we need interactive videos if there are ordinary ones?

The non-linear storyline of the video and the ability to choose the continuation yourself allow the viewer to engage, and content creators will be able to show unique stories, the plot of which will be influenced by users. Also, content creators, by studying the statistics of transitions to videos, will be able to understand what the audience is most interested in and make the materials more attractive.

When developing the project, the guys were inspired by the interactive film Bandersnatch from Netflix, which received a lot of views and good reviews. When the MVP had already been written, the news appeared that Youtube was also planning to launch a platform for interactive series, which once again confirms the popularity of this direction.

MVP includes: interactive player, video maker, content and tag search, video collections, comments, views, ratings, channel and user profiles.

Mail.ru Group tech issue 2019

Technology stack used in the project:

Mail.ru Group tech issue 2019

How the project is planned to be developed:

  • collecting statistics and infographics about video transitions;
  • notifications and private messages for site users;
  • versions for Android and iOS.

After that it is planned to add:

  • create video stories from your phone;
  • editing downloaded video fragments (for example, cropping);
  • creation and launch of interactive advertising in the player.

Project team: Maxim Morev (fullstack developer, project architecture) and Roman Maslov (fullstack developer, project design).

On Line On Board

Mail.ru Group tech issue 2019

Mail.ru Group tech issue 2019

Today, parents pay great attention to the mental development of children, and children are interested in intellectual games. Therefore, chess is gaining popularity again. And although chess is quite popular in general, it is problematic to find a regular opponent for games. Therefore, many people use online chess services, despite the fact that many players prefer to play "live", with real pieces. However, when playing chess, a person puts a lot of mental effort and gets tired, and this fatigue is complemented by the negative impact of sitting at a computer or smartphone. As a result, the brain is overloaded after only two games.

All these factors pushed the authors to the idea of ​​the On-Line-On-Board project, which consists of three parts: a physical chessboard, a desktop application and a web service. The board is an ordinary chess field that recognizes the position of the pieces and, using light indication, indicates the moves of the opponent. The board connects via USB to a PC and communicates with the desktop application. In training mode (and for kids) your possible moves are highlighted.

The application takes over the main functions of managing the board, which can greatly reduce its cost and bring the implementation of most functions to the software level. The application communicates with a web service, the main value of which is dynamic updating.

The main scenario for using the product: one person plays on the service, the second person plays on a physical board connected to the service. That is, the service takes on a communicative function.

Project team: Daniil Tuchin, Anton Dmitriev, Sasha Kuznetsov.

You can read more about our educational projects at this link. And visit the channel more often Technostream, there are regularly new educational videos about programming, development and other disciplines.

Source: habr.com

Add a comment