Hello, colleagues.
Today we present the translation of an article by Tugberk Ugurlu, who has summarized the principles of modern software system design in a comparatively concise manner. Hereâs what the author shares about himself in brief:
Since it is absolutely impossible to cover such a colossal topic as architectural patterns + design patterns as of 2019 in a single hub article, we recommend not only the text by Mr. Ugurlu, but also the numerous links he kindly provided within it. If you enjoy it, we can publish a more specialized text on the design of distributed systems.
Snapshot from Unsplash
If you have never faced challenges such as designing a software system from scratch, starting such work can sometimes leave you uncertain about where to begin. I believe that it's important to outline the boundaries first, so you have a clear idea of what exactly you are going to design, and thenâroll up your sleeves and work without stepping outside those boundaries. As a starting point, you can take a product or service (ideally one that you really like) and examine its implementation. You might be surprised at how simple this product seems and how immense the complexity hidden within it truly is. Remember: , and thatâs okay.
I think the best advice I can give to those starting to design a system is: do not make any assumptions! From the very beginning, you need to clarify the facts known about this system and the expectations associated with it. Here are a few good questions whose answers will help you get started with the design:
- What is the problem we are trying to solve?
- What is the peak number of users who will interact with our system?
- What data reading and writing patterns will we be using?
- What are the expected failure cases, and how do we plan to address them?
- What expectations are there regarding the system's consistency and availability?
- Are there any requirements related to external auditing and regulation that need to be considered?
- What types of confidential data are we planning to store?
These are just a few questions that have been useful in my work, as well as in the teams I have participated in over the years of my professional career. If you know the answers to these questions (and any others relevant to the context in which you work), you can gradually delve into the technical details of the task.
Setting the baseline
What I mean by 'baseline' here? In our times, most problems in the software industry can be 'resolved' using existing methods and technologies. Therefore, by orienting yourself in this landscape, you gain an advantage when facing problems that others have encountered before you. Remember that programs are written to solve business and user problems, so we strive to solve tasks in the most straightforward and simple way (from the user's perspective). Why is this worth remembering? Perhaps in your coordinate system, you prefer finding unique solutions for every task, considering, 'What kind of programmer would I be if I follow patterns everywhere?' In reality, the art lies in making decisions about where and what to do.Of course, each of us occasionally faces unique problems, each of which is a true challenge. However, if our baseline is clearly defined, we know where to direct our efforts: either to search for ready-made solutions to the task at hand or to further explore it and gain a deeper understanding.
I think I have convinced you that if an expert confidently understands the architectural components of some outstanding software systems, these insights will be invaluable for mastering the art of architecture and building a solid foundation in this field.
So, where to start? has a GitHub repository titled , from which you can learn how to design large-scale systems and prepare for interviews on this topic. The repository has a section with examples of , where, in particular, it discusses how they approach the design of their systems , such as Twitter, Uber, etc.
However, before diving into this material, let's take a closer look at the most important architectural challenges commonly faced in practice. This is crucial as it requires specifying A LOT of aspects of an intricate and multifaceted problem, and then addressing it within the constraints of the regulations that govern the system in question. , a former employee of Facebook, recorded , where he shared his own experiences from reviewing hundreds of applicants. While the video explicitly focuses on designing large systems and the success criteria important when selecting candidates for such positions, it will nonetheless serve as a comprehensive resource on what aspects are most important in system design. I also offer of this video.
Build knowledge on data storage and retrieval
Typically, your decision on how you will store and retrieve your data over the long term critically affects system performance. Therefore, you must first understand the expected read and write characteristics of your system. Next, you need to be able to assess these metrics and make choices based on those assessments. However, you can only effectively manage this task if you are familiar with the existing data storage patterns. Essentially, this implies a solid understanding regarding .
Databases can be thought of as data structures that possess exceptional scalability and durability. Therefore, knowledge of data structures should be quite beneficial when choosing a particular database. For example, â is a data structure server that supports various types of values. It allows working with data structures like lists and sets, reading data using well-known algorithms such as , organizing this work in a durable and highly available manner.
Snapshot from Unsplash
Once you have sufficiently oriented yourself in various data storage patterns, you should proceed to study data consistency and availability. The first thing you will need to grasp at least in general terms, and then refine this knowledge by taking a closer look at established patterns of and . This way, you will broaden your understanding in this area and realize that reading and writing data are actually two very distinct problems, each associated with its own particular challenges. Armed with several patterns ensuring consistency and availability, you can significantly enhance system performance while guaranteeing uninterrupted data flow to your applications.
Finally, in concluding the discussion about data storage issues, it is necessary to mention caching. Should it be performed both client-side and server-side? What data will you keep in the cache? And why? How will you organize cache invalidation? Will it be performed regularly, at certain intervals? If so, how often? I recommend starting to explore these topics with the of the aforementioned systems design primer.
Communication Patterns
Systems consist of various components; these can be different processes running on the same physical node or different machines operating in various parts of your network. Some of these resources within your network may be private, but others must be public and accessible to external consumers.
It is necessary to ensure communication between these resources and also to exchange information between the entire system and the outside world. In the context of systems design, we again face a set of new unique challenges. Letâs delve into how can be beneficial, and what.
Snapshot from Unsplash
When organizing communication with the outside world, it is always very important , which also needs to be approached with seriousness and actively managed.
Connection Distribution
I'm not sure that bringing this topic to a standalone section will seem justified to everyone. Nevertheless, I will elaborate on this concept here, and I believe the material in this section is best described by the term 'connection distribution'.
Systems are formed by correctly connecting multiple components, and their communication with each other is often organized based on established protocols, such as TCP and UDP. However, these protocols alone are often insufficient to meet the needs of modern systems, which are frequently operated under heavy loads and are highly dependent on user requirements. It is often necessary to find ways to distribute connections to handle such high loads in the system.
At the core of this distribution lies the well-known (DNS). This system allows for the conversion of a domain name, such as a weighted round robin algorithm and delay-based methods, which help distribute the load.
is fundamentally important, and virtually any major system on the Internet that we encounter today is located behind one or more load balancers. Load balancers help distribute client requests across multiple available instances. Load balancers can be hardware-based or software-based, but in practice, we more often deal with software ones, such as and . conceptually are also very similar to load balancers; however, there are a number of . These differences must be taken into account when designing a system according to your needs.
It is also important to know about (CDN). A CDN is a global distributed network of proxy servers that delivers information from nodes that are geographically closer to a specific user. CDNs are preferable when working with static files written in JavaScript, CSS, and HTML. Additionally, today there is a growing trend for cloud services that provide traffic managers, such as , providing you with global distribution and reduced latency when working with dynamic content. However, such services are usually helpful in cases where you need to work with stateless web services.
Let's talk about business logic. Structuring business logic, task flows, and components.
So, we've discussed various infrastructure aspects of the system. Most likely, the user isnât even aware of all these elements in your system and, honestly, doesn't care about them at all. The user is interested in how it feels to interact with your system, what can be achieved by doing things this way, and how the system executes user commands, what it does with user data.
As is clear from the title of this article, I intended to discuss software architecture and system design. Thus, I didnât plan to cover software design patterns, which describe how software components are created. However, the more I reflect on this, the more I realize that the boundary between software design patterns and architectural patterns is very blurred, and these two concepts are closely related. For instance, letâs take a look at Once you adopt this architectural pattern, it will affect almost every aspect of your system: long-term data storage, the level of consistency adopted in your system, the outlines of components within it, etc. Therefore, I decided to mention some architectural patterns that directly relate to business logic. Even if this article will only provide a simple list, I recommend you familiarize yourself with it and contemplate ideas associated with these patterns. Here you go:
- Concepts , particularly, ,
Collaborative approaches
It is highly unlikely that you will be the only participant responsible for the system design process on a project. On the contrary, you will most likely need to interact with colleagues working both within and outside your task. In this case, you may need to evaluate the chosen technological solutions together with your colleagues, identify business needs, and understand how to best parallelize tasks.
Snapshot from Unsplash
First and foremost, you will need to develop a precise and widely accepted understanding of the business goal you are trying to achieve and the moving parts you will have to deal with. Group modeling techniques, particularly, help to significantly speed up this process and increase your chances of success. You can start this work before or after you outline , and then delve deeper as the product matures. Based on the level of consensus achieved here, you can also formulate a for the limited context in which you are working. When you need to discuss your system's architecture, the , proposed by , can be particularly useful, especially when determining how deeply you need to dive into the details of the issue, visualizing the things you want to convey.
There might also be other mature technologies in this area that are just as useful as domain-driven design. However, we keep returning to the understanding of the domain, so knowledge and experience in the field of will be helpful.
Source: habr.com
