Recently, another winter defense of graduates of our three techno projects took place - Technopark (MSTU named after Bauman), Technosphere (Lomonosov Moscow State University) and Technotrack (MIPT). The teams presented both the implementation of their own ideas and solutions to real business problems proposed by different departments of the Mai.ru Group.
Among the projects:
- Service for the sale of gifts with augmented reality.
- A service that aggregates promotions, discounts and offers from the mailing list.
- Visual search for clothes.
- Service for electronic bookcrossing with the possibility of rent.
- Smart food scanner.
- Modern audio guide.
- Mail.ru Tasks project
- Mobile TV of the future.
We want to tell you more about the six projects that were especially highlighted by the jury members and mentors.
Visual clothing search
The project is presented by a team of Technosphere graduates. According to analysts, the fashion market in Russia in 2018 amounted to almost 2,4 trillion rubles. The guys have created a service that is positioned as an intelligent assistant for shopping in a huge variety of goods. This is a B2B solution that extends the functionality of online stores.
During UX testing, the authors of the project found out that by “similar dress” people mean similarity not in color or pattern, but in clothing attributes. Therefore, the guys developed a system that does not just compare two pictures, but understands the semantic similarity. You upload an image of the item of clothing you are interested in, and the service selects products that are relevant to its attributes.
Technically, the system works as follows:
For detection and classification, the Cascade Mask-RCNN neural network was trained. To determine the attributes and similarity of clothing, a neural network based on ResNext-50 is used with several heads for groups of attributes, and Triplet loss for photos of a single product. The entire project was implemented on the basis of microservice architecture.
In the future it is planned:
- Launch the service for all categories of clothing.
- Develop an API for online stores.
- Improve attribute manipulation.
- Learn to understand queries in natural language.
Project team: Vladimir Belyaev, Petr Zaidel, Emil Bogomolov.
Mobile TV of the future
Project of the Technopark team. Students have created an application with a TV schedule for the main Russian digital broadcasting channels, to which the function of viewing channels using IPTV (online channels) or an antenna has been added.
The most difficult thing was to connect the antenna to the Android device: for this, a tuner was used, for which the authors themselves wrote a driver. As a result, we got the opportunity to watch TV and use the TV guide in one application on Android.
Project team: Konstantin Mitrakov, Sergey Lomachev.
A service that aggregates promotions, discounts and offers from the mailing list
This is a project at the intersection of advertising and postal technologies. Our mailboxes are full of spam and mailing lists. Every day we receive letters with personal discounts, but we open them less and less, perceiving them as “useless advertising”. Because of this, users lose profit and advertisers suffer losses. A study by Mail.ru Mail showed that users want to see a summary of their discounts.
Project collects information about discounts and promotions from your mailing list and shows them in the form of a tape of cards from which you can go to the promotion website or in a letter. The program can work with several mailboxes at once. There is a list of selected stocks.
The project has a microservice architecture and consists of three main parts:
- OAuth authorization for convenient mailbox connection.
- Collection and analysis of emails with promotions.
- Storage and display of cards with discounts.
The project uses natural language processing technology using GPU resources: graphics accelerators made it possible to increase the processing speed by 50 times. The algorithm is based on a question and answer system, which allows you to quickly add stock categories in accordance with new business requirements.
This team won not only a place in the top teams according to the jury, but also won the Digital Peaks 2019 competition. This is a competition of Russian developers who create IT tools to improve the efficiency of business and government structures, as well as to increase personal productivity. Our team won in the student nomination.
The students have big plans for the further development of the project, the next ones:
- Integration with mail services.
- Implementation of an image analysis system.
- Launch of the project for a wide audience.
Project team: Maxim Ermakov, Denis Zinoviev, Nikita Rubinov.
Separately, we want to tell you about three teams that were noted by Mail.ru Group mentors who worked with students during the entire semester. When choosing projects, special attention was paid to the complexity of the project, implementation and teamwork.
Mail.ru Tasks project
The project was noted by both the jury and mentors.
Mail.ru Tasks is the first stand-alone to-do list service developed by the company. In the coming months, Tasks will replace task lists in Mail.ru Calendar, and once the project is enabled for all users, it will be integrated into Mail.ru mobile and web Mail.
The project was implemented using Offline-first and Mobile-first approaches. That is, you can use the web application anytime, anywhere and on anything. Internet access does not matter: the data will be saved and synchronized. For more convenience, you can "install" the application from the browser, and it will look like a native one.
Smart Food Scanner
In the grocery store, we cannot always quickly determine whether a food product is right for us or not, how safe and useful it is. The situation becomes more complicated if a person has dietary restrictions, various allergies, or he is on a diet. The Foodwise Android app allows you to scan a product's barcode and effortlessly see if it's worth it.
use it.
The application has three main sections: "Profile", "Camera" and "History".
In the "Profile" you set your preferences: in the "Ingredients" section, you can exclude any of the 60 ingredients listed in the database from the diet and read information about E-supplements. "Groups" allow you to exclude a whole block of ingredients at once. For example, if you specify "Vegetarian", then all products containing meat will be highlighted in red.
In the "Camera" section, there are two modes: barcode scanning and vegetable and fruit recognition. After scanning the barcode, you will receive all information about the product. The ingredients you have excluded will be highlighted in red.
The "History" will save all previously scanned products. This section is equipped with text and voice search.
The mode of recognition of fruits and vegetables allows you to get information about their nutritional and energy value. For example, one apple contains approximately 25 grams.
carbohydrates, which is unacceptable for people on a low-carbohydrate diet.
The application is written in Kotlin, and the Camera uses the ML Kit to scan barcodes and identify fruits and vegetables. The backend consists of two services: a server API with a database, in
which stores 60 ingredients and compositions of 000 products, as well as a neural network written in Python and Tensorflow.
Project team: Artyom Andryukhov, Ksenia Glazacheva, Dmitry Salman.
Service for the sale of gifts with augmented reality
Every person has received symbolic gifts at least once in their life. Often, the very fact of attention is more important for people than the gift received. Such gifts are not useful, but their production and disposal have a negative impact on the nature of our planet. So the authors of the project came up with the idea to create a service for the sale of gifts with augmented reality.
To test the relevance of the idea, conducted a study. 82% of respondents faced the problem of choosing a gift. For 57% of respondents, the main difficulty in choosing was the fear that their gifts would not be used. 78% of people are willing to change to solve environmental problems.
The authors put forward three theses:
- Gifts live in the virtual world.
- They don't take up space.
- Always near.
To implement augmented reality on the web, the authors chose the AR.js library, which consists of two main parts:
- The first is responsible for drawing graphics on top of the camera stream using A-Frame or Three.js.
- The second part is ARToolKit, which is responsible for recognizing a marker (a special character that can either be printed or displayed on the screen of another device) in the camera's output stream. The marker is used to position the graphics. The presence of ARToolKit does not allow you to create markerless augmented reality using AR.js.
AR.js hides many pitfalls. For example, using it in conjunction with A-Frame can break styles throughout the site. Therefore, the authors used the "bundle" AR.js + Three.js, thanks to which they managed to solve some of the problems. And in order to embed AR.js based on Three.js in React, on which the project site is written, I had to make an AR-Test-2 repository (), which implements a separate React component for using AR.js based on Three.js. Viewing the model in augmented reality and in 3D (for devices without a camera) was implemented.
However, later it turned out that users do not understand what a marker is and how to use it. Therefore, the authors switched to technology , which is currently being actively developed by Google. It uses ARKit (iOS) or ARCore (Android) to display AR models without a marker. The technology is based on Three.js and includes a 3D model viewer. The usability of the app has been greatly improved, however, to view augmented reality, you need a device with iOS 12 or later.
The project is now available at) where you can get your first gift.
Project team: Denis Stasiev, Anton Chadov.
You can read more about our educational projects at . And visit the channel more often , there are regularly new educational videos about programming, development and other disciplines.
Source: habr.com