The DBOS (DBMS-oriented Operating System) project is presented, developing a new operating system to run scalable distributed applications. A feature of the project is the use of a DBMS to store applications and the state of the system, as well as organizing access to the state only through transactions. The project is being developed by researchers from the Massachusetts Institute of Technology, Wisconsin and Stanford Universities, Carnegie Mellon University, and Google and VMware. The developments are distributed under the MIT license.
Components for interacting with hardware and low-level memory management services are moved to the microkernel. The capabilities provided by the microkernel are used to run the DBMS layer. High-level system services that provide application execution interact only with a distributed DBMS and are separated from the microkernel and system-specific components.
Building on top of a distributed DBMS allows you to make system services initially distributed and not tied to a specific node, which distinguishes DBOS from traditional cluster systems, in which each node runs its own instance of the operating system, on top of which separate cluster schedulers, distributed FS and network managers are launched.
It is noted that the use of modern distributed DBMSs that store data in RAM and support transactions, such as VoltDB and FoundationDB, as the basis for DBOS, can provide sufficient performance for the efficient execution of many system services. The DBMS can also host scheduler, file system, and IPC data. At the same time, DBMSs are highly scalable, provide atomicity and isolation of transactions, can manage petabytes of data, provide tools for managing access and tracking data flows.
Among the advantages of the proposed architecture is a significant expansion of analytics capabilities and a reduction in code complexity due to the use of ordinary requests to the DBMS in the operating system services, on the side of which the implementation of transactions and tools for ensuring high availability is taken out (such functionality can be implemented on the side of the DBMS once and used in OS and applications).
For example, a cluster scheduler can store information about tasks and processors in DBMS tables and implement scheduling operations as normal transactions, mixing imperative code and SQL. The use of transactions simplifies problems such as multithreading and crash recovery, because transactions guarantee the consistency and persistence of state. In the context of the scheduler example, transactions enable concurrent access to shared data and ensure that state integrity is maintained in the event of failures.
The logging and data analysis mechanisms provided in the DBMS can be used to track access and change the state of applications, monitor, debug and maintain security. For example, after detecting unauthorized access to the system, it is possible to determine the extent of the leak by executing SQL queries, revealing all the operations performed by processes that have gained access to confidential information.
The project has been developed for more than a year and is at the stage of creating prototypes of individual components of the architecture. At present, a prototype of operating system services working on top of the DBMS, such as FS, IPC, and the scheduler, has been prepared, and a software environment is being developed that provides an interface for running applications based on the FaaS (function-as-a-service) model.
At the next stage of development, it is planned to provide a full-fledged software stack for distributed applications. VoltDB is still used as a DBMS in the experiments, but discussions are underway on creating our own layer for storing data or implementing missing features in existing DBMSs. The question of which components should be executed at the kernel level and which can be implemented on top of the DBMS is also under discussion.
Source: opennet.ru