{"id":144566,"date":"2025-10-09T17:11:54","date_gmt":"2025-10-09T15:11:56","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/sistema-szhatiya-openzl-operezhayushhaya-zstd-i-xz-po-skorosti-i-urovnyu-szhatiya-strukturirovannyh-dannyh"},"modified":"2025-10-09T17:11:54","modified_gmt":"2025-10-09T15:11:56","slug":"sistema-szhatiya-openzl-operezhayushhaya-zstd-i-xz-po-skorosti-i-urovnyu-szhatiya-strukturirovannyh-dannyh","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/news\/sistema-szhatiya-openzl-operezhayushhaya-zstd-i-xz-po-skorosti-i-urovnyu-szhatiya-strukturirovannyh-dannyh","title":{"rendered":"The OpenZL compression system outperforms Zstd and XZ in speed and compression level of structured data.","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Meta* has introduced the OpenZL toolkit for data compression and decompression, demonstrating a higher compression level and speed compared to Zstd and XZ formats. OpenZL is designed for efficiently compressing structured datasets, such as those used in machine learning, as well as storage containing fields with various repetitive types of information. OpenZL is written in C\/C++ and is open-sourced under the BSD license. <\/p>\n

When compressing a database with the astronomical star catalog SAO, the OpenZL toolkit achieved a data size reduction of 2.06 times, while the zstd algorithm compressed the information by 1.31 times and XZ by 1.64 times. In terms of compression speed, OpenZL surpassed zstd by twice (203 MB\/s vs. 115 MB\/s) and XZ by 65 times (203 MB\/s vs. 3.1 MB\/s). The decompression speed in OpenZL was slightly slower than zstd (822 MB\/s vs. 890 MB\/s) and 27 times faster than XZ.

\"The<\/center> <\/p>\n

OpenZL is not a general-purpose algorithm and performs well only for data with a predefined structure. The operation of OpenZL involves adaptive generation of a packer based on the provided data description. As a result, code is generated for compression, optimized for a specific data format. A universal decompressor compatible with all generated packers is used for decompression. <\/p>\n

Packaging and unpacking is performed using a single utility 'zli' or the libopenzl library. The data structure is described in terms of profiles. A set of predefined profiles is already included, which describe standard storage formats. For example, there is a profile for the CSV format or for data stored as an array of 64-bit numbers. Compression involves selecting a profile with the command 'zli list-profiles' and starting the packaging process with the command 'zli compress --profile profile_name'. To unpack, it is enough to run 'zli decompress'. <\/p>\n

For specific formats, it is necessary to create a custom profile using the command 'zli train', which identifies patterns in the data and generates a profile with optimal compression levels. Using the '--pareto-frontier' option, the created profile can be upgraded to speed up packing or unpacking at the cost of reducing compression levels. To describe complex formats with nested structures and to define the layout of data formats within structures, the SDDL (Simple Data Description Language) can be used. <\/p>\n

The method for creating optimal packers is based on using a set of primitive encoders, each of which is most effective for specific types and sequences of data. To compress, a directed acyclic graph of data processing is formed, where the nodes are codecs and the edges represent data variations in the processed format. Depending on the type of data input, a chain of codecs is selected to optimally compress the incoming data item. With this organization, the file header is compressed by one codec, the integer data field by a second, the field with an increasing counter by a third, and the string data field by a fourth.

\"The<\/center>
\n
Source: opennet.ru<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Meta* \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u0438\u043b\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u0439 \u0434\u043b\u044f \u0441\u0436\u0430\u0442\u0438\u044f \u0438 \u0440\u0430\u0441\u043f\u0430\u043a\u043e\u0432\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 OpenZL, \u043f\u043e \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044e \u0441 \u0444\u043e\u0440\u043c\u0430\u0442\u0430\u043c\u0438 Zstd \u0438 XZ \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0449\u0438\u0439 \u0431\u043e\u043b\u0435\u0435 \u0432\u044b\u0441\u043e\u043a\u0438\u0439 \u0443\u0440\u043e\u0432\u0435\u043d\u044c \u0441\u0436\u0430\u0442\u0438\u044f \u0438 \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u0440\u0430\u0431\u043e\u0442\u044b. OpenZL \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u043d \u0434\u043b\u044f \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0433\u043e \u0441\u0436\u0430\u0442\u0438\u044f \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u043d\u0430\u0431\u043e\u0440\u043e\u0432 \u0434\u0430\u043d\u043d\u044b\u0445, \u043d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u043c\u044b\u0445 \u043f\u0440\u0438 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u043c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0438, \u0430 \u0442\u0430\u043a\u0436\u0435 \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449, \u0441\u043e\u0434\u0435\u0440\u0436\u0430\u0449\u0438\u0445 \u043f\u043e\u043b\u044f \u0441 \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u043c\u0438 \u043f\u043e\u0432\u0442\u043e\u0440\u044f\u044e\u0449\u0438\u043c\u0438\u0441\u044f \u0442\u0438\u043f\u0430\u043c\u0438 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u0438. \u041a\u043e\u0434 OpenZL \u043d\u0430\u043f\u0438\u0441\u0430\u043d \u043d\u0430 C\/C++ \u0438 \u043e\u0442\u043a\u0440\u044b\u0442 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":144567,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-144566","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47OpenZL compression system, outperforming Zstd and XZ in speed and compression level for structured data | ProHoster","description":"Meta* has introduced the OpenZL compression and decompression toolkit, which demonstrates a higher level of compression and speed compared to the Zstd and XZ formats.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/news\/sistema-szhatiya-openzl-operezhayushhaya-zstd-i-xz-po-skorosti-i-urovnyu-szhatiya-strukturirovannyh-dannyh","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0421\u0438\u0441\u0442\u0435\u043c\u0430 \u0441\u0436\u0430\u0442\u0438\u044f OpenZL, \u043e\u043f\u0435\u0440\u0435\u0436\u0430\u044e\u0449\u0430\u044f Zstd \u0438 XZ \u043f\u043e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u0438 \u0438 \u0443\u0440\u043e\u0432\u043d\u044e \u0441\u0436\u0430\u0442\u0438\u044f \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 | ProHoster","og:description":"\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Meta* \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u0438\u043b\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u0439 \u0434\u043b\u044f \u0441\u0436\u0430\u0442\u0438\u044f \u0438 \u0440\u0430\u0441\u043f\u0430\u043a\u043e\u0432\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 OpenZL, \u043f\u043e \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044e \u0441 \u0444\u043e\u0440\u043c\u0430\u0442\u0430\u043c\u0438 Zstd \u0438 XZ \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0449\u0438\u0439 \u0431\u043e\u043b\u0435\u0435 \u0432\u044b\u0441\u043e\u043a\u0438\u0439 \u0443\u0440\u043e\u0432\u0435\u043d\u044c \u0441\u0436\u0430\u0442\u0438\u044f \u0438 \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u0440\u0430\u0431\u043e\u0442\u044b.","og:url":"https:\/\/prohoster.info\/en\/blog\/news\/sistema-szhatiya-openzl-operezhayushhaya-zstd-i-xz-po-skorosti-i-urovnyu-szhatiya-strukturirovannyh-dannyh","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2025-10-09T15:11:56+00:00","article:modified_time":"2025-10-09T15:11:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"144566","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-23 15:15:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2026-01-23 15:15:19","updated":"2026-01-23 15:15:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/144566","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=144566"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/144566\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/144567"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=144566"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=144566"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=144566"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}