Facebook (banned in Russia) has published the Sapling source code management system used in the development of the company's internal projects. The system aims to provide a familiar versioning interface that can scale to very large repositories spanning tens of millions of files, commits and branches. The client code is written in Python and Rust, and is open under the GPLv2 license.
Separately, a server part was developed for effective remote work with repositories and a virtual file system for working with a local slice of a part of the repository as with a full repository (the developer sees the entire repository, but only the requested data is copied to the local system, which are accessed). The code for these components used in the Facebook infrastructure is not yet open, but the company promised to publish it in the future. However, prototypes of the Mononoke server (in Rust) and VFS EdenFS (in C++) can already be found in the Sapling repository. These components are optional and the Sapling client is enough to work, which supports cloning Git repositories, interacting with servers based on Git LFS, and working with git hosts such as GitHub.
The main idea of the system is that when interacting with a special server part that provides storage of the repository, all operations are scaled depending on the number of files actually used in the code the developer is working on, and do not depend on the total size of the entire repository. For example, a developer may use only a small portion of code from a very large repository, and only this small portion, rather than the entire repository, will be transferred to his system. The working directory is filled dynamically, as files from the repository are accessed, which, on the one hand, allows you to significantly speed up work with your part of the code, but on the other hand, leads to a slowdown when you first access new files and requires constant access to the network (separately provided and offline-mode of preparation of commits).
In addition to adaptive data loading, Sapling also implements optimizations aimed at reducing the loading of information with a history of changes (for example, 3/4 of the data in a repository with the Linux kernel is history of changes). To work effectively with the change history, the data associated with it is stored in a segmented view, which allows you to download separate parts of the commit graph from the server. The client can ask the server for information about the relationship of several commits and download only the necessary part of the graph.
The project has been developed over the past 10 years and was created to solve problems when organizing access to very large monolithic repositories with one master branch, in which the practice of using the "rebase" operation instead of "merge" was practiced. At that time, there were no open solutions for working with such repositories, and Facebook engineers decided to create a new version control system that meets the needs of the company, instead of splitting projects into small repositories, which would lead to more complicated dependency management (at one time, to solve a similar problem, Microsoft created GVFS layer). Initially, Facebook used the Mercurial system and the Sapling project was initially developed as an addition to Mercurial. Over time, the system transformed into an independent project with its own protocol, storage format and algorithms, which was also extended with the ability to interact with Git repositories.
For work, the “sl” command-line utility is proposed, which implements typical concepts, workflows, and an interface familiar to developers familiar with Git and Mercurial. The terminology and commands in Sapling are slightly different from Git and closer to Mercurial. For example, “bookmarks” are used instead of branches (named branches are not supported), by default, when performing clone / pull, not the entire repository is loaded, but only the main branch, there is no preliminary marking of commits (staging area), instead of “git fetch” the command “sl” is used pull", instead of "git pull" - "sl pull --rebase", instead of "git checkout COMMIT" - "sl goto COMMIT", instead of "git reflog" - "sl journal", to revert the change instead of "git checkout - FILE" "sl revert FILE" is specified, and "." is used to identify the "HEAD" branch. But in general, the general concepts of branches and clone/pull/push/commit/rebase operations are preserved.
Among the additional features of the Sapling toolkit, support for a “smart log” (smartlog) stands out, which allows you to visually assess the state of your repository, highlight the most important information and filter out minor details. For example, when you run the sl utility without arguments, only your own local changes are displayed (foreign ones are collapsed), the status of external branches, changed files, and new versions of commits are shown. Additionally, an interactive web-interface is offered, which makes it possible to quickly navigate through the smart log, change tree and commits.
Another notable improvement in Sapling is the simplification of the process of fixing and parsing errors, and reverting to a previous state. For example, the commands "sl undo", "sl redo", "sl uncommit" and "sl unamend" are offered to rollback many operations, "sl hide" and "sl unhide" commands to temporarily hide commits, and interactive navigation through old states and return to the specified point the command "sl undo -i command". Sapling also supports the concept of a commit stack, which allows you to organize a step-by-step review by breaking down complex functionality into a set of smaller and more understandable incremental changes (from a basic framework to a finished feature).
Several add-ons have been prepared for Sapling, including the ReviewStack interface for reviewing changes (code under GPLv2), which allows you to process pull requests on GitHub and use a stack view of changes. In addition, additions have been published for integration with VSCode and TextMate editors, as well as the implementation of the ISL (Interactive SmartLog) interface and server.
Source: opennet.ru