Almost four years ago, we started designing a brand new distributed mail system called Mailion, which is designed for corporate communications. Our solution is built on Cloud Native microservice architecture, capable of working with more than 1 users at the same time and will be ready to cover 000% of the needs of large corporations.
During the work on Mailion, the team has grown several times, and now almost 70 developers are involved in the product. We have come a long way from the idea and the first prototypes to the stage of piloting the commercial version. It's time to tell Habr about what kind of product we are creating, how our mail system is arranged and works, what technology stack we use and why our solution is the future of corporate communications. Let's go!
Habr, hello! My name is Anton Gerasimov, I am the head of the development department at the company's Moscow development center . Today we want to present Mailion, a fundamentally new Russian corporate-class mail system that will become a worthy alternative to popular foreign solutions. Mailion offers high load capacity, unparalleled scalability and resiliency, and requires minimal attention from system administrators.
Now our development is in the beta stage, but very soon, according to our plan, by the end of 2020, it will move into the status of a pilot implementation of a commercial product.
For the most part, this article contains general information - it is simply impossible to cover a complex software product in one publication. I plan to make a series of articles with a story about key technologies. In the meantime, for your convenience, I offer the following content:
What is a corporate mail system?
The simple and obvious answer to this question is the email and calendar tool. But the devil, as you know, is in the details.
So, we are creating a new generation of mail with advanced scheduling functions, which is focused on the corporate segment and large companies. We are focused on working with large commercial and government structures, which can have from 30 thousand to several hundred thousand jobs.
Our system is based on the principle of a platform solution. We build on the concept of Unified Communications and offer a single tool for working with mail, calendar and documents within one system, with the ability to increase the functionality of the solution by connecting additional modules and components. Extensions, by the way, can be developed not only by us, but also by our partners. A fairly wide SDK toolkit will also be published for this purpose.
Who needs another mail and why?
Large business has formed an interesting trend - enterprises are experiencing a need for collaboration features. This is not only about the direct interaction of employees through the transfer of something, but about sharing access to mail and role boxes, the ability to work with large groups of participants and the presence of various corporate functions.
A typical corporate function is the email recall mechanism, which is in high demand in large companies, but is practically not used either in the B2C segment or in small businesses. The thing is that among small groups of users, the likelihood of such a need is rather small, and the cost of a mistake is much less. Moreover, it is simply impossible to implement this function outside the corporate mail system - even in Google mail there is no response of letters in the form that Exchange users are used to. The root of the problem is in the design of mail protocols from the 80s of the last century.
We've conducted dozens of focus groups, interviewed hundreds of users, and spent several thousand man-hours identifying typical mail usage scenarios. This allowed us to carefully study the needs of our potential users and the peculiarities of their business processes. Based on the data received, we decided to develop an architecture that would allow, if necessary, to scale up to the required load volume. In our view, there is no limit at all, but at the moment we understand how to ensure the work of companies that employ several hundred thousand users.
What is hidden under the hood
Our product includes an email solution, a scheduling tool, an address book, and interactive document viewing based on our own MyOffice document management solution.
Since we are talking about a corporate-level solution, a full-fledged search engine is also an integral part of corporate mail. Our solution is capable of conducting end-to-end morphological search across all components. In addition, the entire mail system is based on its own storage, which is also optimized for collaboration.
What are the differences between MyOffice mail systems
The reader of Habr, who already had experience with MyOffice solutions, knows that MyOffice Mail is present as part of commercial products. And the question arises - what is its difference from the corporate mail system Mailion, which my team worked on?
We at MyOffice decided that to meet the needs of companies of various sizes, it is necessary to make two enterprise-class mail systems. The MyOffice Mail product is intended for organizations with thousands or several tens of thousands of users. For larger structures, it is more expedient to use Mailion, a new MyOffice product, which is built on Cloud Native microservice architecture, has unprecedented scalability and fault tolerance.
When choosing products, customers need to decide in advance on the strategy for the further development of their company. For any organization, implementing a mail system is a complex project that does not involve quickly replacing one solution with another. Therefore, the maximum possible load, fault tolerance and self-healing criteria, as well as the geographical distribution factor should be taken into account in advance.
The two MyOffice mail systems are fundamentally different. They are built using different technology stacks, and the tools for each of the systems are selected in accordance with the goals and objectives of the segment in which the mail system will be deployed. We have a common product vision of what should be present in mail systems, as well as synchronization in design approaches. But these are two products that are designed for different audiences of users.
What are the challenges facing developers
Further in the text, I will only talk about the new corporate mail system Mailion.
We have conducted a thorough research and analyzed the advantages and disadvantages of existing email solutions. The accumulated experience of our specialists, who in the past were directly involved in the creation of high-load systems, allowed us to quickly identify the main pain point of any well-known product - the performance of disk input and output (IO).
We can say that the task of accelerating IO operations has become the main challenge that we faced. It was necessary to focus on solving our specific tasks, and we began to create and develop our own binary data storage system. This approach has a very obvious economic component - we need not only to store data, but also to perform a lot of disk input and output operations.
Such a decision was not easy. We looked at a lot of enterprise-class products that are aimed at storing information, and could not find anything ready and suitable for our requirements. After all, we need not only to store data, but also to constantly work with them, and not in exclusive mode with access to one user, but to ensure the possibility of collaboration of several tens of thousands of users.
So, we have determined that the most priority corporate criterion for us is working with large amounts of data.
Reliability
Moreover, this is not so much a matter of storage as the task of processing and quick access to this information. It is required to ensure high reliability indicators - the time of continuous operation of the entire solution should be at the level of 99,9%. However, the mail system must be autonomous in making the right decisions about bounce handling. Only in this way it is possible to achieve the absence of failures that can paralyze the business processes of a particular company.
fault tolerance
We have worked out a set of measures that are applied to each component, and selected approaches to ensure fault tolerance. In particular, the corporate mail system uses data quorum, service reservation and statelessness control, as well as its own routing and consensus-based quorum mechanisms.
The criterion for high fault tolerance is the presence in each component of mechanisms for self-testing. Mailion constantly analyzes whether this or that function is executed correctly or incorrectly, whether this or that block works correctly. This is one of the means of self-healing of the system in case of failure. Information should not be lost, the solution should go into a consistent state eventually.
It must be understood that it is not possible to foresee all the probable events in life in general - you cannot deceive physics. But we adhere to such design approaches that allow us to speak with a fairly high degree of confidence about the correct handling of accidents of certain patterns. We also have a large testing team that is constantly trying to break something in the product. And sometimes they succeed. Based on this fact, an incident is started, which we study in detail and, on its basis, create a new mechanism to eliminate such problems.
How the system reacts to crashes
The market often asks the question - what happens to the system if it suddenly loses connection with one of its nodes? Such a request is especially relevant if you plan to implement a geographically distributed solution.
When designing such an installation, it is necessary to take into account the needs of the customer and his set of reliability criteria. It is a mistake to assume that the connection to the data network is the same everywhere. In corporate systems, even the speed of the signal through the fiber is important to consider.
If the customer makes high demands on reliability, then we will recommend installation in several data centers, which will be connected by dedicated communication lines.
Nevertheless, the scenario of a complete power outage in one of the nodes of the mail system cannot be ruled out. In this case, work with the master data of this segment will be suspended, but the rest of the parts will work as if nothing had happened. The system will notify administrators of the loss of a node, and employees will need to take a number of appropriate actions.
Also, our mail system can be configured in such a way that reservations take place inside the data center. This approach will ensure operability in the event of the loss of several nodes inside the data center, and no one will notice the loss of one of the nodes - except, of course, the monitoring system. When such an event occurs, the load is simply redistributed. Therefore, when designing a mail system, it is important to take into account the margin - how many and which components can be "lost" in an accident.
Independence
Absolutely reliable systems do not exist, and failures one way or another, but can happen. Therefore, when designing a mail system, it is important to provide the ability to quickly localize failures, which will allow them to be eliminated within a certain SLA period. Moreover, failures should be eliminated by the system on its own, while minimizing the intervention of system administrators and maintenance services.
Ease of operation
Of course, large systems cannot but require the attention of highly qualified employees. But one of the aspects that we consider when designing is that the system should be easy to administer and not require any unique competencies.
Cost-effectiveness
Of course, to work with any such enterprise systems, training of employees is required - after all, we are talking about really huge scales. Therefore, when creating our product, we focused on reducing the total cost of ownership of the system. For any business, it is important to monitor the efficiency of spending and strive to avoid the endless injection of money into operating expenses.
Decentralization
To achieve the principles I spoke about earlier, it is necessary that the system be completely distributed - a centralized approach is always limited by the conditions of use of the hardware.
Of course, you can buy one big server and try to put all the applications and services on it. But the larger the system, the more complex the whole design, and the reliability of such a system is steadily declining - after all, one server means a single point of failure. And the more we centralize work, the more we depend on that central point.
System Requirements
When we talk about complex corporate mail systems, we need to understand that there is no such thing as a “minimal configuration for work”. At the heart of any large implementation is always serious design work. It allows you to correlate the needs of a particular business and the optimal configuration of the mail system.
The corporate mail system is a flexible tool that can be customized according to a large number of parameters. We conduct load testing according to our own methodology, which allows us to simulate the load based on the analysis of a dozen basic and several hundred possible criteria.
We can say that when preparing an implementation project, it is the calculations that will show the intensity of the work of users in the existing business processes of the organization. At the same time, many input parameters are analyzed, and, depending on the possible load, a decision is made on the required amount of computing power - from one computer to an entire computer room.
Therefore, talking about some minimal configurations is not entirely correct. In particular, we also have a demo stand that runs on just one machine. But we understand that work in the corporate segment requires installation on dozens and even hundreds of virtual machines.
Technology stack
When creating products, we pay special attention to the license purity of the code. High quality standards are set and enforced at MyOffice, which require us to fully develop products ourselves, with minimal involvement of external code.
Much of Mailion's code is in-house development, code that is wholly owned by us and that we can change and modify as needed. Most of the code for our mail system is self-written in Go (Golang). In addition to Go, we use C++ as well as Java Script ES6 for the web part.
The remaining 5% are so-called "heavy components" such as databases. These include RethinkDB, ArangoDB, and Redis. Of the key technologies, I also note gRPC - a remote procedure call system that is used as a single mechanism for interacting via API, this is an important part.
What is the product made of
A corporate mail system is not a “server in a vacuum”. Our product includes about 70 components and 45 services that support the mail system. All these elements are written from scratch and are MyOffice's own development.
The most visible part of the system is the server complex, which provides storage, mail, calendar and search systems, and a web client. We are also developing several user applications and plan to offer our users new "thin clients" based on our web client technologies by the time of the commercial release.
Mail system compatibility
Mailion works according to standard protocols. In order to start using our solution, you just need to set up client applications and continue working with them in a familiar environment. For example, you can use the client application included with MyOffice Mail.
Also, for the convenience of our users, we have created a special plugin that provides a seamless transition from Exchange. With its help, users will be able to continue working with MS Outlook when replacing the mail server - they will not even notice that something has changed in their usual interaction with communication tools. Such a plugin allows you to smoothly transfer users from the old mail system, without stopping the work of the entire organization. This is especially important if you need to replace tools for hundreds of thousands of users.
Ease of administration
A feature of our corporate mail system is the need for extremely little participation of administrators. When designing, we tried to find a balance between the possibility of customization and the complexity of management. To achieve this, we needed to conduct a lot of research on the behavior patterns of administrators of complex corporate systems.
As I said earlier, our team consists of highly qualified specialists - each of them has solid administration experience. Therefore, when developing the system, we tried to transfer their experience to the needs of users when managing a complex product. In other words, knowing about potential problems, we foresaw ways to solve them at the architectural design level in advance.
For example, the
- isolation of system components into containers, including infrastructure containers, is provided - this helps us increase the level of security, flexibly configure access between components, and also allows us to quickly include new items in the list of supported systems and platforms that are in demand by business;
- a single approach to development and deployment, a single configuration format for all components contribute to a more understandable and efficient work and reduce support costs;
- own system installer, based on the usual deployment tools, allows you to centrally, flexibly and understandably manage the installation;
microservice architecture with a clear division of responsibility of business logic between services helps to understand the scheme of the system and more efficiently provide support; - the infrastructure part of the system is based on familiar solutions, using best practices, and also allows for integration with existing systems in the enterprise, which simplifies the commissioning of the system within the company;
- for this purpose, an artificial intelligence-based system will be provided in the storage facility, which will be able to independently identify potential failures and manage equipment.
Modern design
It took several thousand man-hours to create the product design. Before the development, the task was to make the interface modern - easy and intuitive. It's tricky when you're designing an enterprise system that needs to include a number of specific features by default.
Of course, design cannot be "carved in stone" or approved for signature and seal - it is a living tool, it is constantly evolving and improving. Over the course of four years, we have transformed the design several times, but there has not been a radical change in ideology. The key principle of building the entire corporate product line is to create a universal design system.
When designing the design of a corporate mail system, we paid special attention to the development of three key areas - adaptability, consistency, and the ability to adapt to the client's corporate identity.
Responsiveness
A design feature of the interfaces of the Mailion corporate mail system is the ability to display on devices with screens of any size. All visual components are carefully drawn, they scale correctly when working on mobile devices and are perfectly suited even for finger control on touch screens. Also, when the size of the application window is changed, the column layout is also transformed - the size of the columns is shifted proportionally, in strict accordance with the postulates of the concept of rich Internet applications.
Consistency
The design of Mailion corporate mail system is based on a systematic approach. We have been developing this direction since the beginning of 2017, when it was not as fashionable to talk about design systems as it is now. A design system is a set of rules and tools for visual and technical execution that reflects the philosophy of a product and is constantly evolving. We introduced it to simplify communication between development and design.
Corporate identity customization
Our design is a "chameleon" that can adapt to any client's corporate identity. The colors of all interface elements are defined through variables. Thus, by changing the basic set of several colors, you can recolor all applications at once.
Fonts are also defined through variables: the interface can use the corporate font of the client company, if it is provided for by the client's brand book. At the same time, different fonts can have different letter sizes, and we take this into account. For example, PT Sans in size 16 looks noticeably smaller than Roboto in the same size 16. Therefore, we adjust the size of text blocks to the features of a particular font.
We will devote separate articles to the structure of our color library and the peculiarities of working with fonts.
Mobile Apps
Mobile applications are an integral part of any modern mail system - users should be able to work from anywhere and at any time.
Now we are just forming the concept of corporate mobile applications, they should take into account the differences in the functionality of the MyOffice mail systems and offer different approaches to the use of e-mail and calendar systems in corporations.
Intelligent media panel
Our focus groups showed that users find the interfaces of familiar email systems cumbersome. The respondents we interviewed noted that working with calendar events and attachments was particularly difficult. We tried to achieve a responsive interface and increase productivity - to make our product pleasant to use.
The media panel, a new collaboration tool, organizes all information about a conversation and provides instant access to the list of participants in the conversation, links and versions of documents that have ever been sent.
This tool helps in situations where users have to work with large tracks of hundreds of posts. It often happens that you need to find a document or picture that was in only one of them. The media panel creates a list of all the items in the email thread and displays them in one place. As a result, finding the desired object becomes much easier.
Localization
The need for a corporate mail system is not only among Russian users. MyOffice pursues a consistent policy of entering foreign markets, so the issues of product localization are given quite a lot of attention.
In our product, the need for localization into foreign languages was laid down from the very beginning, at the level of the basic part of the system. In itself, adding new languages is not a big problem for us - this is a completely understandable task that we know how to solve.
Now Russian and English languages are available to users. In the next releases, we will add support for the languages in which MyOffice products work — French, Spanish, Portuguese, and others — to the corporate mail system. In some respects, it is easier for us than for colleagues from neighboring departments, since for mail and calendar systems it is only necessary to take into account the alphabet and the peculiarities of working with dates and times - fortunately, we do not have formulas, as in a spreadsheet editor.
Yes, we are hiring!
It took several hundred man-years to create our product. And with all my desire, I would not be able to tell about everything at once within the framework of one article. Nevertheless, I hope that this publication will serve as a starting point for getting to know our product - as I said above, I plan to continue to talk in more detail about both the solution itself and its features, as well as our approaches to development.
Without a shadow of modesty, I will also say that today the number of companies that develop such mail systems around the world is a few. This is a really complex engineering task, it requires both a deep understanding of the needs of corporate customers, and a thorough analysis of the business processes of large organizations, modern trends in design and development, as well as a fair amount of competent specialists. Our mail system is evolving every day.
Right now we are almost open vacancies in development. Come to work with us if you want to work with us to create a product that can change the way the corporate world thinks about email.
Source: habr.com