This article's translation was prepared in anticipation of the course launch .
Key Points:
- It is crucial to develop a schema even though it is optional in MongoDB.
- Similarly, indexes should match your schema and access patterns.
- Avoid using large objects and big arrays.
- Be cautious with MongoDB settings, especially regarding security and reliability.
- There's no query optimizer in MongoDB, so you need to be careful when executing query operations.
I've been working with databases for a long time, but I only recently discovered MongoDB. There are a few things I wish I had known before starting with it. When someone already has experience in a certain field, they have preconceived notions about what databases are and what they do. Hoping to ease the understanding for others, I present a list of common mistakes.
Creating a MongoDB server without authentication
Unfortunately, MongoDB is installed without authentication by default. For a workstation that is accessed locally, this practice is acceptable. But since MongoDB is a multi-user system that tends to use large amounts of memory, it would be better to install it on a server with the maximum possible RAM under your circumstances, even if you plan to use it only for development. Installation on a server through the default port can be problematic, especially if any JavaScript code can be executed in the query (for example, $where as an idea for ).
There are several authentication methods, but the easiest way is to set up a user ID/password. Use this idea while you consider a more elaborate authentication based on . When it comes to security, MongoDB should be updated regularly, and logs should always be checked for unauthorized access. Personally, I like to choose a different port as the default.
Don't forget to bind the attack surface to MongoDB
contains good advice for reducing the risk of network breaches and data leaks. It's easy to dismiss and say that a server for development doesn’t require a high level of security. However, it’s not that simple, and it applies to all MongoDB servers. In particular, unless there’s a compelling reason to use , or , you should disable the use of arbitrary code in JavaScript by writing in the configuration file . Since standard MongoDB data files are not encrypted, it’s sensible to run MongoDB with , who has full access to the files, with limited access only for them and the ability to use their own file access control methods from the operating system.
Error in schema design
MongoDB does not use a schema. But that doesn't mean a schema isn't needed. If you just want to store documents without any consistent schema, you can save them quickly and easily, but retrieving them later can be .
The classic article " is worth reading, and features such as in the third-party tool Studio 3T are worth using for regular schema checks.
Don't forget about sorting order
Neglecting the sorting order can be the biggest disappointment and waste more time than using any other incorrect configuration. By default, MongoDB uses . But it's unlikely to be useful to anyone. Case-sensitive, accent-sensitive, binary sorts were considered curious anachronisms along with beads, kaftans, and curling mustaches back in the 1980s. Now their use is inexcusable. In real life, "motorcycle" is the same as "Motorcycle." And "Britain" and "britain" refer to the same place. A lowercase letter is simply the uppercase equivalent. And don't get me started on diacritic sorting. When creating a database in MongoDB, use sorting options that ignore accents and , which correspond to the language and . This significantly simplifies searching string data.
Creating collections with large documents
MongoDB is pleased to accommodate large documents up to 16 MB in collections, while is designed for larger documents exceeding 16 MB. However, just because large documents can be stored there does not mean it's the best idea. MongoDB performs optimally when you save individual documents sized in kilobytes, treating them more like rows in a wide SQL table. Large documents will lead to issues with .
Creating documents with large arrays
Documents can contain arrays. It's best if the number of elements in an array is far from four digits. If elements are frequently added to the array, it may exceed the document containing it, necessitating , which in turn requires . When re-indexing a document with a large array, indexes are often rewritten, as there is a , storing its index for each element. Such re-indexing also occurs when a document is inserted or deleted.
MongoDB has something called , which provides space for document growth, minimizing this issue.
You might think that you can do without indexing arrays. Unfortunately, the absence of indexes can lead to other problems. Since documents are scanned from beginning to end, searching for elements at the end of the array will take longer, and most operations associated with such a document will be .
Remember that the order of stages in aggregation matters
In a database system with a query optimizer, the queries you write are explanations of what you want to obtain, not how to obtain it. This mechanism works analogously to ordering in a restaurant: typically, you simply order a dish rather than giving detailed instructions to the chef.
In MongoDB, you instruct the chef. For instance, you need to ensure that data passes through reduce as early as possible in the pipeline using $match and $project, with sorting occurring only afterward reduce, and that the search occurs exactly in the order you need. Having a query optimizer that eliminates unnecessary work, optimally organizes steps, and selects the type of connection can spoil you. In MongoDB, you gain more control at the cost of convenience.
Tools like simplify the construction of aggregation queries in . The Aggregation Editor feature allows you to apply pipeline operators one step at a time, as well as check input and output data at each step for easier debugging.
Using fast write
Never set high-speed write settings in MongoDB that offer low reliability. This mode "file-and-forget" seems fast since the command returns before the write is completed. If the system crashes before the data is written to disk, it will be lost and remain in an inconsistent state. Fortunately, 64-bit MongoDB includes logging.
The MMAPv1 and WiredTiger storage engines use logging to prevent this, though WiredTiger can recover to the last consistent , if logging is disabled.
Logging ensures that the database is in a consistent state after recovery and retains all data up to the point of logging. The frequency of writes is configured using the parameter .
To ensure reliable writes, make sure logging is enabled in the configuration file ), and the frequency of writes matches the amount of information you can afford to lose.
Sorting without an index
When searching and aggregating, there is often a need to sort data. We hope this is done at one of the final stages, after filtering the results to reduce the amount of data being sorted. And even in that case, you will need . You can use a single or compound index.
If an appropriate index is not available, MongoDB will do without it. There is a memory limit of 32 MB on the total size of all documents in , and if MongoDB reaches this limit, it will either throw an error or return .
Searching without index support
Search queries serve a function similar to the JOIN operation in SQL. For optimal performance, they require an index on the key value used as a foreign key. This is not obvious, as the use is not reflected in explain(). Such indexes are an addition to the index recorded in explain(), which in turn is used by pipeline operators $match and $sort, when they appear at the beginning of the pipeline. Indexes can now encompass any stage of the .
Abandoning the use of multi-updates
Element.getAnimations() is used to modify part of an existing document or the entire document, up to full replacement depending on the parameter you set . It is not so obvious that it will not process all documents in the collection until you set the parameter to update all documents matching the query criteria.
Do not forget the importance of key order in a hash table
In JSON, an object consists of an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.
Unfortunately, BSON attaches great importance to order when searching. In MongoDB, the order of keys within embedded objects , i.e. { firstname: "Phil", surname: "factor" } is not the same as { { surname: "factor", firstname: "Phil" }. Therefore, you must maintain the order of name/value pairs in documents if you want to be sure you can find them.
Do not confuse "null" and "undefined"
Value "undefined" which has never been valid in JSON, according to JSON (ECMA-404, Section 5), even though it is used in JavaScript. Furthermore, for BSON it is deprecated and converted to $null, which is not always a good solution. .
Using $limit() without $sort()
Very often, when you are developing in MongoDB, it is useful just to see a sample of the results that will return from a query or aggregation. For this task, you will find $limit(), but it should never be present in the final version of the code unless you are using it before. $sortThis mechanism is necessary because otherwise you cannot guarantee the order of the results and will not be able to reliably view the data. At the top of the results, you will receive different entries depending on the sorting. For reliable operation, queries and aggregations must be deterministic, meaning they should yield the same results for each execution. Code that contains $limit(), but does not contain $sort, will not be considered deterministic and may later lead to errors that are difficult to trace.
Conclusion
The only way to be disappointed with MongoDB is to compare it directly with another type of database, such as a relational database management system, or to approach its use with certain expectations. It's like comparing an orange to a fork. Database systems pursue specific goals. It is best to simply understand and appreciate these differences for yourself. It would be a shame to pressure MongoDB developers because of the path that forced them to go down the relational database route. I want to see new and interesting ways to solve old problems, such as ensuring data integrity and creating data systems resistant to failures and attacks from malicious actors.
The implementation of ACID transactionality in MongoDB version 4.0 is a good example of introducing significant improvements in an innovative way. Multi-document and multi-operator transactions are now atomic. There is also the ability to control the time it takes to acquire locks and terminate hanging transactions, as well as to change the isolation level.
Read more:
Source: habr.com
