Modern information systems are quite complex. Their complexity is largely due to the intricate data being processed within them. The complexity of the data often lies in the variety of data models employed. For example, when data becomes 'big', one of the inconveniences is not just its volume but also its variety.

If you don't yet find a flaw in the reasoning, read on.

Content

Polyglot persistence
Multimodality
Multimodel databases based on the relational model
     Document model in MS SQL Server
     Graph model in MS SQL Server
Multimodel databases based on the document model
     Relational model in MarkLogic
     Graph model in MarkLogic
Multimodel databases 'without a primary model'
     ArangoDB
     OrientDB
     Azure CosmosDB
Multimodel databases based on the graph model?
Conclusion
Survey

Polyglot persistence

The aforementioned leads to the need to use several different databases within even a single system for data storage and addressing various processing tasks, each supporting its own data model. This concept, popularized by M. Fowler, author numerous well-known books and one of the co-authors of the Agile Manifesto, has come to be known as polyglot persistence. (‘polyglot persistence’).

Fowler also provides the following example of data storage organization in a fully functional and high-load application within the e-commerce sector.

This example is, of course, somewhat exaggerated, but there are reasonable considerations for choosing a specific database for the corresponding purpose, such as here.

It is clear that being a keeper in such a zoo is not easy.

The amount of code responsible for data storage grows proportionally with the number of databases used; the amount of code synchronizing data increases, ideally not proportionally to the square of that number.
The costs of ensuring enterprise characteristics (scalability, fault tolerance, high availability) multiply with each database employed.
It is impossible to ensure enterprise characteristics for the storage subsystem as a whole, especially transactionality.

From the perspective of the zoo director, it looks like this:

The costs for licenses and support from the database vendor rise exponentially.
Expansion of the team and lengthening of deadlines.
Direct financial losses or penalties due to data inconsistencies.

There is a significant increase in the total cost of ownership (TCO) of the system. Is there a way out of the 'multi-model storage' situation?

Multimodality

The term 'multi-model storage' entered common usage in 2011. Recognizing the problems of the approach and finding solutions took several years, and by 2015, Gartner analysts articulated the answer:

From the 'Market Guide for NoSQL DBMSs — 2015»:
The future of DBMSs, their architectures, and usage methods — multi-modality.
From the 'Magic Quadrant for ODBMS — 2016»:
Leading operational DBMSs will offer multiple models — relational and non-relational — within a single platform.

It seems that this time Gartner analysts were not mistaken in their predictions. If you visit the page with the main ranking of DBMSs on DB-Engines, you can see that thehigher level of isolation, as if one controller is broken, the problem is confined to that specific context).majority of its leaders position themselves as multi-model DBMSs. The same is true for any specific ranking page.

The table below lists the DBMSs that are leaders in each specific ranking, claiming their multi-modality. For each DBMS, the originally supported model (once the only one) is indicated alongside the models currently supported. Also listed are DBMSs that position themselves as 'originally multi-model', without any claims of an initial inherited model.

DBMS	Original model	Additional models
Oracle	Relational	Graph, document
MS SQL	Relational	Graph, document
PostgreSQL	Relational	Graph*, document
MarkLogic	Document	Graph, relational
MongoDB	Document	Key-value, graph*
DataStax	Wide-column	Document, graph
Redis	Key-value	Document, graph*
ArangoDB	—	Graph, document
OrientDB	—	Graph, document, relational
Azure CosmosDB	—	Graph, document, relational

Notes to the table

Statements marked with asterisks in the table require clarifications:

The PostgreSQL DBMS does not support the graph data model, however, such a product based on it, such as AgensGraph.
In relation to MongoDB, it is more accurate to speak of the existence of graph operators in the query language ($lookup, $graphLookup), than about support for the graph model, although, of course, their introduction required some optimizations at the physical storage level towards supporting the graph model.
In relation to Redis, this refers to the extension RedisGraph.

Next, for each of the classes, we will show how support for multiple models is implemented in database management systems of this class. We will consider relational, document, and graph models to be the most important, and demonstrate with examples from specific DBMS how the 'missing' models are implemented.

Multimodel databases based on the relational model

The leading DBMS today are relational; Gartner's forecast would not have been considered fulfilled if RDBMS had not shown movement towards multimodality. And they are showing it. Now, the notion that a multimodal DBMS is like a Swiss Army knife, for which nothing can be done well, can be directed straight at Larry Ellison.

However, the author prefers the implementation of multimodality in Microsoft SQL Server, on the example of which support for document and graph models in RDBMS will be described.

Document model in MS SQL Server

Regarding how support for the document model is implemented in MS SQL Server, there have already been two excellent articles on Habr, so I will limit myself to a brief summary and commentary:

The way to support the document model in MS SQL Server is quite typical for relational DBMS: JSON documents are proposed to be stored in regular text fields. Support for the document model includes providing special operators for parsing this JSON:

JSON_VALUE for extracting scalar values of attributes,
JSON_QUERY for extracting subdocuments.

The second argument of both operators is an expression in a JSONPath-like syntax.

Abstractly, it can be said that documents stored in this way are not 'first-class entities' in a relational DBMS, unlike tuples. Specifically, in MS SQL Server, there are currently no indexes on the fields of JSON documents, which complicates table join operations based on these field values and even fetching documents by these values. However, it is possible to create a computed column on such a field and index it.

Additionally, MS SQL Server provides the ability to conveniently construct a JSON document from the contents of tables using the operator FOR JSON PATH — a capability that, in a sense, is opposite to the previous approach of conventional storage. It is clear that no matter how fast the DBMS is, this approach contradicts the ideology of document-based DBMSs, which essentially store ready-made answers to popular queries, and can only address development convenience issues, not performance.

Finally, MS SQL Server allows for solving the problem that is the opposite of document construction: you can decompose JSON into tables using OPENJSON. If the document is not entirely flat, it will require using CROSS APPLY.

Graph model in MS SQL Server

Support for the graph (LPG) model is also implemented in Microsoft SQL Server quite predictably: it is recommended to use special tables for storing nodes and for storing the edges of the graph. Such tables are created using expressions CREATE TABLE AS NODE and CREATE TABLE AS EDGE respectively.

Tables of the first type are similar to regular tables for storing records, with the only external distinction being that the table contains a system field $node_id — a unique identifier for the graph node within the database.

Similarly, tables of the second type have system fields $from_id and $to_id, and records in such tables clearly define the connections between nodes. Separate tables are used to store relationships of each type.

Let's illustrate this with an example. Suppose graph data has a schema as shown in the provided diagram. Then, to create the corresponding structure in the database, the following DDL queries need to be executed:

CREATE TABLE Person (
  ID INTEGER NOT NULL,
  name VARCHAR(100)
) AS NODE;

CREATE TABLE Cafe (
  ID INTEGER NOT NULL, 
  name VARCHAR(100), 
) AS NODE;

CREATE TABLE likes (
  rating INTEGER
) AS EDGE;

CREATE TABLE friendOf
  AS EDGE;

ALTER TABLE likes
  ADD CONSTRAINT EC_LIKES CONNECTION (Person TO Cafe);

The main specificity of such tables is that graph patterns with Cypher-like syntax can be used in queries to them (however, features like “*” and others are not yet supported). Based on performance measurements, it can also be assumed that the data storage method in these tables differs from the mechanism used for storing data in regular tables and is optimized for executing such graph queries.

SELECT Cafe.name
  FROM Person, likes, Cafe
  WHERE MATCH (Person-(friendOf)-(likes)->Cafe)
  AND Person.name = 'John';

Moreover, it is quite challenging to avoid using these graph patterns when working with such tables, as conventional SQL queries would require additional effort to retrieve the system's 'graph' identifiers of nodes for similar tasks ($node_id, $from_id, $to_id; for the same reason, data insertion queries are not included here as they would be overly cumbersome).

In summary of the implementations of document and graph models in MS SQL Server, I would note that having one model layered over another does not seem effective primarily from a language design perspective. One language needs to be expanded by another; the languages are not entirely 'orthogonal', and the compatibility rules can be quite whimsical.

Multimodel databases based on the document model

In this section, I would like to illustrate the implementation of multimodality in document databases using the example of one of the less popular ones, MongoDB (as mentioned, it has only conditionally graph operators $lookup and $graphLookup, which do not work on sharded collections), but rather with a more mature and 'enterprise' database system. MarkLogic.

So, let's assume the collection contains a set of XML documents of the following type (MarkLogic also allows storing JSON documents):

John
  Smith

Relational model in MarkLogic

The relational representation of the document collection can be created using a mapping template (with the contents of elements value in the example below being any arbitrary XPath):

/Person
  
    
      Person
      
        
          SSN
          @SSN
          string
        
        
          name
          name
        
        
          surname
          surname

An SQL query can be directed at the created view (for example, via ODBC):

SELECT name, surname FROM Person WHERE name="John"

Unfortunately, the relational representation created via the mapping template is read-only. When processing queries against it, MarkLogic will attempt to use document indexes.Previously, MarkLogic also had limited relational representations, fully based on indexes and writable, but they are now considered deprecated.

Graph model in MarkLogic

With support for the graph (RDF) model, things work out about the same. Again, using a mapping template you can create an RDF representation of the document collection from the example above:

/Person
    
      
        PREFIX
        "http://example.org/example#"
      
    
  
    
      sem:iri( $PREFIX || @SSN )
      sem:iri( $PREFIX || surname )
      
    
    
      sem:iri( $PREFIX || @SSN )
      sem:iri( $PREFIX || name )

A SPARQL query can be addressed to the resulting RDF graph:

PREFIX : 
SELECT ?name ?surname {
  :631803299804 :name ?name ; :surname ?surname .
}

Unlike the relational model, MarkLogic supports the graph model in two other ways:

The DBMS can be a fully separate RDF data store (the triples in it will be called managed as opposed to those described above extracted).
RDF in special serialization can simply be inserted into XML or JSON documents (and the triples will then be called unmanaged). This is likely an alternative to the mechanisms idref and so on.

A good understanding of how things are "actually" arranged in MarkLogic is provided by Optic API, in this sense, it is low-level, although its purpose is rather the opposite — to try to abstract from the data model used, ensure consistent operation with data across various models, transactional integrity, and so on.

Multimodel databases 'without a primary model'

There are also DBMSs on the market that position themselves as initially multimodel, having no inherited core model. They include ArangoDB, OrientDB (since 2018, the developer company has belonged to SAP) and CosmosDB (a service within the Microsoft Azure cloud platform).

In fact, the "core" models in ArangoDB and OrientDB do exist. In both cases, they are their own data models that generalize the document model. Generalizations primarily concern easing the ability to perform graph and relational queries.

These models are the only ones available for use in the specified DBMS, and their own query languages are designed to work with them. Certainly, such models and DBMS are promising, but the lack of compatibility with standard models and languages makes it impossible to use these DBMS in legacy systems — replacing the already used DBMS there with them.

There was already a wonderful article about ArangoDB and OrientDB on Habr: JOIN in NoSQL databases.

ArangoDB

ArangoDB claims support for the graph data model.

Nodes in the graph in ArangoDB are regular documents, and the edges are documents of a special type, having system fields along with regular system fields (_key, _id, _rev) system fields _from and _to. Documents in document DBMS are traditionally grouped into collections. The collections of documents representing edges in ArangoDB are called edge collections. By the way, documents in edge collections are also documents, so edges in ArangoDB can also act as nodes.

Source Data

Let us consider a collection persons, whose documents look like this:

[
  {
    "_id"  : "people/alice" ,
    "_key" : "alice" ,
    "name" : "Alice"
  },
  {
    "_id"  : "people/bob" ,
    "_key" : "bob" ,
    "name" : "Bob"  
  }
]

Let there also be a collection cafes:

[
  {
    "_id" : "cafes/jd" ,
    "_key" : "jd" ,
    "name" : "John Donne"  
  },
  {
    "_id" : "cafes/jj" ,
    "_key" : "jj" ,
    "name" : "Jean-Jacques"
  }
]

Then the collection likes could look like this:

[
  {
    "_id" : "likes/1" ,
    "_key" : "1" ,
    "_from" : "persons/alice" ,
    "_to" : "cafes/jd",
    "since" : 2010 
  },
  {
    "_id" : "likes/2" ,
    "_key" : "2" ,
    "_from" : "persons/alice" ,
    "_to" : "cafes/jj",
    "since" : 2011 
  } ,
  {
    "_id" : "likes/3" ,
    "_key" : "3" ,
    "_from" : "persons/bob" ,
    "_to" : "cafes/jd",
    "since" : 2012 
  }
]

Queries and results

A graph-style query in the AQL language used in ArangoDB, returning a human-readable account of which café each person likes, looks like this:

FOR p IN persons
  FOR c IN OUTBOUND p likes
  RETURN { person : p.name , likes : c.name }

In relational style, where we rather 'compute' relationships than store them, this query can be rewritten as (by the way, without the collection likes it could have been avoided):

FOR p IN persons
  FOR l IN likes
  FILTER p._key == l._from
    FOR c IN cafes
    FILTER l._to == c._key
    RETURN { person : p.name , likes : c.name }

The result will be the same in both cases:

[
  { "person" : "Alice" , likes : "Jean-Jacques" } ,
  { "person" : "Alice" , likes : "John Donne" } ,
  { "person" : "Bob" , likes : "John Donne" }
]

More queries and results

If it seems that the output format above is more characteristic of a relational DBMS than a document one, you can try this query (or alternatively, use COLLECT):

FOR p IN persons
  RETURN {
    person : p.name,
    likes : (
      FOR c IN OUTBOUND p likes
      RETURN c.name
    )
}

The result will look like this:

[
  { "person" : "Alice" , likes : ["Jean-Jacques" , "John Donne"]  } ,
  { "person" : "Bob" , likes : ["John Donne"] }
]

OrientDB

The implementation of the graph model over the document model in OrientDB is based on of document fields having, in addition to more or less standard scalar values, values of types such as LINK, LINKLIST, LINKSET, LINKMAP and LINKBAG. The values of these types are references or collections of references to system identifiers documents.

The identifier assigned by the system has a "physical meaning", indicating the record's position in the database, and looks something like this: @rid : #3:16. Thus, the values of reference properties are actually more like pointers (as in the graph model), rather than selection conditions (as in relational models).

As in ArangoDB, in OrientDB edges are represented as separate documents (although if the edge has no properties, it can be made lightweight, and there will not be a separate document corresponding to it).

Source Data

In a format close to the dump format of the OrientDB database, the data from the previous example for ArangoDB would look something like this:

[
     {
      "@type": "document",
      "@rid": "#11:0",
      "@class": "Person",
      "name": "Alice",
      "out_likes": [
        "#30:1",
        "#30:2"
      ],
      "@fieldTypes": "out_likes=LINKBAG"
    },
    {
      "@type": "document",
      "@rid": "#12:0",
      "@class": "Person",
      "name": "Bob",
      "out_likes": [
        "#30:3"
      ],
      "@fieldTypes": "out_likes=LINKBAG"
    },
    {
      "@type": "document",
      "@rid": "#21:0",
      "@class": "Cafe",
      "name": "Jean-Jacques",
      "in_likes": [
        "#30:2",
        "#30:3"
      ],
      "@fieldTypes": "in_likes=LINKBAG"
    },
    {
      "@type": "document",
      "@rid": "#22:0",
      "@class": "Cafe",
      "name": "John Donne",
      "in_likes": [
        "#30:1"
      ],
      "@fieldTypes": "in_likes=LINKBAG"
    },
    {
      "@type": "document",
      "@rid": "#30:1",
      "@class": "likes",
      "in": "#22:0",
      "out": "#11:0",
      "since": 1262286000000,
      "@fieldTypes": "in=LINK,out=LINK,since=date"
    },
    {
      "@type": "document",
      "@rid": "#30:2",
      "@class": "likes",
      "in": "#21:0",
      "out": "#11:0",
      "since": 1293822000000,
      "@fieldTypes": "in=LINK,out=LINK,since=date"
    },
    {
      "@type": "document",
      "@rid": "#30:3",
      "@class": "likes",
      "in": "#21:0",
      "out": "#12:0",
      "since": 1325354400000,
      "@fieldTypes": "in=LINK,out=LINK,since=date"
    }
  ]

As we can see, the vertices also store information about incoming and outgoing edges. When used With the Document API, you must monitor referential integrity yourself, while the Graph API takes care of this for you. But let's see how a query to OrientDB looks in pure, non-integrated query languages.

Queries and results

A query analogous to the one in the ArangoDB example for OrientDB looks like this:

SELECT name AS person_name, OUT('likes').name AS cafe_name
   FROM Person
   UNWIND cafe_name

The result will be returned as follows:

[
  { "person_name": "Alice", "cafe_name": "John Donne" },
  { "person_name": "Alice", "cafe_name": "Jean-Jacques" },
  { "person_name": "Bob",  "cafe_name": "Jean-Jacques" }
]

If the format of the result still seems overly "relational", you need to remove the line with UNWIND():

[
  { "person_name": "Alice", "cafe_name": [ "John Donne", "Jean-Jacques" ] },
  { "person_name": "Bob",  "cafe_name": [ "Jean-Jacques" ] }
]

The query language of OrientDB can be characterized as SQL with Gremlin-like inserts. In version 2.2, a Cypher-like query form was introduced, MATCH :

MATCH {CLASS: Person, AS: person}-likes->{CLASS: Cafe, AS: cafe}
RETURN person.name AS person_name, LIST(cafe.name) AS cafe_name
GROUP BY person_name

The result format will be the same as in the previous query. Consider what needs to be removed to make it more "relational", as in the very first query.

Azure CosmosDB

To a lesser extent, the above applies to ArangoDB and OrientDB regarding Azure CosmosDB. CosmosDB provides the following APIs for data access: SQL, MongoDB, Gremlin, and Cassandra.

The SQL API and MongoDB API are used to access data in the document model. The Gremlin API and Cassandra API are used to access data in the graph and column models, respectively. Data in all models is stored in the internal model format of CosmosDB: ARS ("atom-record-sequence"), which is also close to the document model.

However, the data model chosen by the user and the API used are fixed at the time of account creation in the service. It is not possible to access data uploaded in one model in the format of another model, which could be illustrated by something like this:

Thus, multimodality in Azure CosmosDB today represents merely the ability to use multiple databases supporting different models from the same provider, which does not resolve all issues related to multi-variant storage.

Multimodel databases based on the graph model?

It is worth noting that there are currently no multimodal DBMS with a graph model at their core (excluding multimodality supporting two graph models simultaneously: RDF and LPG; see more about this in previous publication). The greatest difficulties arise from implementing a document-based model on top of a graph model, rather than a relational one.

The question of how to implement a relational model on top of a graph model was considered back in the early days of the latter. As said, for example, David McGovern:

There is nothing inherent in the graph approach that prevents creating a layer (e.g., by suitable indexing) on a graph database that enables a relational view with (1) recovery of tuples from the usual key value pairs and (2) grouping of tuples by relation type.

When implementing a document model on top of a graph, one should bear in mind, for example, the following:

JSON array elements are considered ordered, whereas those originating from the top of a graph edge are not;
Data in a document model is usually denormalized; storing multiple copies of the same nested document is generally undesirable, and subdocuments usually do not have identifiers;
On the other hand, the ideology of document-oriented databases is that documents are ready-made 'aggregates' that do not need to be rebuilt each time. It is necessary to ensure that the graph model allows for quick retrieval of the subgraph corresponding to the ready document.

A bit of advertising

The author of the article is involved in the development of the NitrosBase database system, whose internal model is graph-based, while its external models — relational and document-based — are representations of it. All models are equal: practically any data is accessible in any of them using their respective query languages. Moreover, data can be modified in any representation. Changes will be reflected in the internal model and, accordingly, in other representations.

I will describe how the models correspond in NitrosBase, hopefully in one of the next articles.

Conclusion

I hope the general outlines of what is called multimodality have become more or less clear to the reader. Multimodal databases are quite diverse, and 'support for multiple models' can look different. To understand what is referred to as 'multimodality' in each specific case, it is useful to answer the following questions:

Is it about supporting traditional models or some single 'hybrid' model?
Are the models 'equal', or is one of them subordinate to the others?
Are the models indifferent to each other? Can data written in one model be read in another or even overwritten?

I think we can now give a positive answer to the question of the relevance of multimodel databases, but it is interesting to consider which of their varieties will be in greater demand in the near future. It seems that multimodel databases supporting traditional models, primarily relational ones, will be more in demand; the popularity of multimodel databases offering new models that combine the advantages of various traditional ones is more of a distant future.

Only registered users can participate in the survey. Please log in, please.