Hello colleagues.
Today, we offer you a translation of an article by Tugberk Ugurlu, who undertook to present the principles of designing modern software systems in a relatively small volume. Here is what the author says about himself in the bottom line:
Since it is absolutely impossible to cover such a colossal topic as architectural patterns + design patterns as of 2019, we recommend not only the text of Mr. Uruglu, but also the numerous links that he kindly placed in it. If you like it, we will publish a more highly specialized text on the design of distributed systems.
Photo from Unsplash website
If you have never had to face such challenges as designing a software system from scratch, then when you start such work, sometimes it is not even clear where to start. I believe that first you need to draw boundaries in order to have a more or less confident idea of ββ\uXNUMXb\uXNUMXbwhat exactly you are going to design, and then roll up your sleeves and work within these boundaries. As a starting point, you can take some product or service (ideally, one that you really like) and figure out how to implement it. You might be amazed at how simple this product looks and how much complexity it actually hides. Do not forget: and that's ok.
I think the best advice I can give to anyone starting out with system design is: don't make any assumptions! From the outset, you need to specify the facts known about this system and the expectations associated with it. Here are some good questions to help you get started designing:
- What is the problem we are trying to solve?
- What is the peak number of users that will interact with our system?
- What patterns of writing and reading data will we use?
- What are the expected failure cases, how are we going to deal with them?
- What are your expectations for system consistency and availability?
- Do you have to take into account any requirements related to external verification and regulation during your work?
- What kinds of sensitive data are we going to store?
These are just a few of the questions that have come in handy both for me and for the teams I have been involved in over the years of my professional career. If you know the answers to these questions (and any others that are relevant to the context in which you have to work), then you can gradually delve into the technical details of the problem.
Set the initial level
What do I mean by "baseline" here? Actually, in our time, most of the problems in the software industry "can" be solved using existing methods and technologies. Accordingly, by navigating this landscape, you get a certain head start when faced with tasks that someone had to solve before you. Keep in mind that programs are written to solve business and user problems, so we strive to solve the problem in the most straightforward and simple (from the user's point of view) way. Why should this be remembered? Maybe you like to look for unique solutions to all problems in your coordinate system, because you think, βwhat kind of programmer am I if I follow patterns everywhereβ? In fact, the art here lies in making decisions about where and what to do. Of course, any of us from time to time have to deal with unique problems, each of which is a real challenge. However, if our initial level is clearly defined, then we know what to spend our energy on: searching for ready-made options for solving the problem set before us, or further studying it and understanding it more deeply.
I think I managed to convince you that if a specialist confidently understands what the architectural component of some wonderful software systems is, then this knowledge will be indispensable for mastering the art of the architect and developing a solid basis in this area.
Okay, so where do you start? At There is a repository on GitHub called , based on which you can learn how to design large-scale systems, as well as prepare for interviews on this topic. The repository has a section with examples , where, in particular, it is considered how they approach the design of their systems e.g. Twitter, Uber, etc.
However, before moving on to this material, let's take a closer look at the most important architectural challenges that one has to face in practice. This is important because you have to specify MANY aspects of an intractable and multifaceted problem, and then solve it within the framework of the regulation that operates in this system. , a former Facebook employee, wrote , where he shared his own experience in reviewing hundreds of applicants. While the video is explicitly about designing large systems and the success criteria that are important when looking for a candidate for such a position, it still serves as a definitive resource on what things are most important when designing systems. I also suggest this video.
Gain knowledge about data storage and retrieval
As a rule, your decision on how you will permanently store and retrieve your data has a critical impact on system performance. Therefore, you must first understand the expected write and read characteristics of your system. Then you need to be able to evaluate these indicators and make a choice based on the assessments made. However, you can only do this job effectively if you understand existing data storage patterns. In principle, this implies certain knowledge related to .
Databases can be thought of as data structures with exceptional scalability and durability. Therefore, knowledge of data structures should be very useful to you when choosing a particular database. For example, is a data structure server that supports various kinds of values. It allows you to work with data structures such as lists and sets, read data using well-known algorithms, for example, organizing such work in a durable and highly accessible style.
Photo from Unsplash website
When you are sufficiently familiar with the various patterns of data storage, move on to studying the consistency and availability of data. First of all, you need to understand at least in general terms, and then polish this knowledge by examining established patterns in more detail ΠΈ . In this way, you will broaden your horizons in this area and understand that reading and writing data are actually two very different problems, and each of them has its own special challenges. Armed with a few consistency and availability patterns, you can significantly increase system performance while ensuring that data flows to your applications without interruption.
Finally, completing the conversation about data storage issues, we should also mention caching. Should it run on the client and server at the same time? What data will be in your cache? And why? How do you organize cache invalidation? Will it be done regularly, at regular intervals? If yes, how often? I recommend that you start working on these topics with the aforementioned primer on systems design.
Communication patterns
Systems are made up of various components; they can be different processes running within the same physical host, or different machines running on different parts of your network. Some of these resources within your network may be private, but others should be public and open to consumers accessing them from outside.
It is necessary to ensure the communication of these resources with each other, as well as the exchange of information between the entire system and the outside world. In the context of systems engineering, here again we are faced with a set of new and unique challenges. Understanding what can be useful , and what r.
Photo from Unsplash website
When organizing communication with the outside world, it is always very important , which also needs to be approached with the utmost seriousness and dealt with actively.
Connection distribution
I'm not sure that putting this topic in an independent section will seem justified to everyone. However, I will elaborate on this concept here, and I believe that the material in this section is most accurately described by the term connection distribution.
Systems are formed by correctly connecting many components, and their communication with each other is often organized on the basis of well-established protocols, such as TCP and UDP. However, these protocols as such are often not enough to meet all the needs of modern systems, which are often operated under high load, and also highly dependent on the needs of users. It is often necessary to find ways to distribute connections in order to cope with such high loads in the system.
This distribution is based on the well-known (DNS). Such a system allows domain name transformations such as weighted round robin and delay-based methods to help balance the load.
is fundamentally important, and almost every large system on the Internet that we have to deal with today is located behind one or more load balancers. Load balancers help distribute client requests across multiple available instances. Load balancers are both hardware and software, however, in practice, you often have to deal with software, for example, ΠΈ . conceptually also very similar to load balancers, although there are a number of . These differences must be taken into account when designing a system based on your needs.
You should also be aware of (CDN). CDN is a global distributed network of proxy servers that delivers information from those nodes that are geographically located closer to a particular user. CDNs are preferred if you are working with static files written in JavaScript, CSS, and HTML. In addition, cloud services that provide traffic managers are common today, for example, , giving you global distribution and reduced latency when working with dynamic content. However, such services are usually useful in cases where you need to work with stateless web services.
Let's talk about business logic. Structuring business logic, task flows and components
So, we managed to discuss various infrastructural aspects of the system. Most likely, the user does not even think about all these elements of your system and, frankly, does not worry about them at all. The user is interested in what it is like to interact with your system, what can be achieved by doing this, and how the system executes user commands, what and how it does with user data.
As the title of this article implies, I was going to talk about software architecture and systems design in it. Accordingly, I did not plan to cover software design patterns that describe how software components are created. However, the more I think about it, the more it seems to me that the line between software design patterns and architectural patterns is very blurred, and the two concepts are closely related. Let's take, for example, (event sourcing). Once you adopt this architectural pattern, it will affect almost every aspect of your system: long-term storage of data, the level of consistency adopted in your system, the shape of the components in it, etc., etc. Therefore, I decided to mention some architectural patterns that are directly related to business logic. Even though this article will have to limit itself to a simple list, I recommend that you familiarize yourself with it and think about the ideas associated with these patterns. Here you are:
- Concepts , in particular, ,
Collaborative approaches
It is highly unlikely that you will be on the project the one participant who is solely responsible for the system design process. On the contrary, you will most likely have to interact with colleagues working both within and outside of your task. In this case, you may need to evaluate the selected technology solutions together with colleagues, isolate business needs and understand how best to parallelize tasks.
Photo from Unsplash website
The first step is to develop an accurate and generally accepted idea of ββwhat the business goal you are trying to achieve is and what moving elements you will have to deal with. Group modeling techniques, in particular, (event storming) help to significantly speed up this process and increase your chances of success. This work can be done before or after you outline and then deepen it as the product matures. Based on the level of consistency that will be achieved here, you can also formulate for the limited context in which you work. When you need to talk about the architecture of your system, for this you may find it useful proposed , especially when you need to understand how much you have to go into the details of the problem by visualizing the things you want to communicate.
There is probably another mature technology in this topic that is no less useful than domain-specific design. However, we somehow return to understanding the subject area, so knowledge and experience in the field should be useful to you.
Source: habr.com