Our team is excited to announce the release of a free open-source monitoring system. !
Is version 4.2 the answer to the ultimate question of life, the universe, and monitoring in general? Let's find out!
As a reminder, Zabbix is a universal system for monitoring the performance and availability of servers, engineering and network equipment, applications, databases, virtualization systems, containers, IT services, and web services.
Zabbix implements a complete cycle from data collection, processing and transformation, analysis of the collected data, to storing, visualizing, and sending notifications using escalation rules. The system also provides flexible options for extending data collection and notification methods, as well as automation capabilities through the API. The single web interface enables centralized management of monitoring configurations and the distribution of access rights to various user groups. The project code is freely distributed under the license. .
Zabbix 4.2 is a new non-LTS version with a shorter official support period. For users looking for a long product lifecycle, we recommend using LTS versions such as 3.0 and 4.0.
So, let's discuss the new features and major improvements in version 4.2:
More official platforms
In addition to the existing official packages, we also offer new builds for:
- RaspberryPi, Mac OS/X, SUSE Enterprise Linux Server 12
- MSI for Windows agent
- Docker images
Built-in support for Prometheus for application monitoring
Zabbix can collect data in various ways (push/pull) from different data sources. This includes JMX, SNMP, WMI, HTTP/HTTPS, RestAPI, XML Soap, SSH, Telnet, agents, scripts, and other sources. And now, meet the support for Prometheus!
Strictly speaking, data collection from Prometheus exporters was possible before thanks to the HTTP/HTTPS data element type and regular expressions.
However, the new version allows for maximum efficiency when working with Prometheus due to its built-in support for the PromQL query language. Using dependent metrics enables the collection and processing of data most effectively: you request the data once, and then distribute it across the necessary metrics.
Getting the value of a specific metric
It is important to note that low-level detection can currently use collected data to automatically generate metrics. In this case, Zabbix converts the received data into JSON format, which is very convenient to work with.
Finding metrics using the filter in the PromQL query language
Currently, there are more than for third-party services and applications with Zabbix. Support for Prometheus will allow us to add a whole range of applications that have official or community-supported Prometheus exporters. This includes monitoring popular services, containers, and cloud resources.
Effective high-frequency monitoring
Do we want to detect problems as quickly as possible? Of course, there is no doubt! Most often, this approach leads to the necessity of polling devices and collecting data too frequently, resulting in a heavier load on the monitoring system. How can we avoid this?
We have implemented a throttling mechanism in preprocessing rules. Throttling essentially allows us to skip identical values.
Suppose we are monitoring the status of a critical application. Every second we check whether our application is functioning or not. In Zabbix, a continuous stream of data of 1 (working) and 0 (not working) comes in. For example: 1111111111110001111111111111…
When everything is fine with our application, Zabbix receives a stream of ones only. Is it necessary to process them? Generally not, since we are only interested in changes in the application’s status; we do not want to collect and store so much data. Thus, throttling allows us to skip a value if it is identical to the previous one. As a result, we will only get data on the change of status, for example, 01010101… This is quite enough information to detect problems!
Ignored values are simply disregarded by Zabbix, they are not recorded in history and do not affect triggers. From Zabbix's perspective, ignored values do not exist.
Ignoring duplicate values
Great! Now we can poll devices very frequently while instantly detecting problems without storing unnecessary information in the database.
But what about the graphs? They will be empty due to lack of data! And how can we tell if Zabbix is collecting data if most of it is being missed?
We thought about that too! Zabbix offers another type of throttling, throttling with heartbeat.
We check once a minute whether the metric is alive
In this case, Zabbix, despite the repeated stream of data, will save at least one value within the specified time interval. If data is collected once a second and the interval is set to one minute, Zabbix will convert a per-second stream of ones into a per-minute stream. It's easy to see that this leads to a 60-fold compression of the collected data.
Now we are sure that data is being collected, the nodata() trigger function works, and the graphs are fine!
Validation of collected data and error handling
None of us want to collect erroneous or unreliable data. For example, we know that the temperature sensor should return data within the range of 0°C to 100°C, and any other value should be considered erroneous and/or ignored.
This is now possible using built-in preprocessing validation rules for compliance or non-compliance with regular expressions, value ranges, JSONPath, and XMLPath.
Now we can control the error response. If the temperature is out of range, we can simply ignore that value, set a default value (for example, 0°C), or define our own error message, such as 'Sensor damaged' or 'Replace the battery.'
Temperature must be between 0 and 100, everything else is ignored
A good example of using validation is the ability to check incoming data for the presence of an error message and set this error for the entire metric. This functionality is very useful when receiving data from external APIs.
Any data transformation using JavaScript
If the built-in preprocessing rules are not enough for us, we now offer complete freedom with the use of arbitrary scripts in JavaScript!
Just one line of code to convert degrees Fahrenheit to degrees Celsius
This opens up limitless possibilities for processing incoming data. The practical benefit of this functionality is that we no longer need external scripts that we used for any operations with data. Now everything can be done using JavaScript.
Now data transformation, aggregation, filtering, arithmetic and logical operations, and much more are possible!
We extract useful information from the Apache mod_status output!
Testing preprocessing
We no longer have to guess how our complex preprocessing scenarios work. There is now a convenient check for the correctness of preprocessing directly from the interface!
Processing millions of metrics per second!
Until Zabbix 4.2, preprocessing was solely handled by the Zabbix server, which limited the opportunities for using proxies to distribute the load.
Starting from Zabbix 4.2, we achieve incredibly efficient load scaling by supporting preprocessing on the proxy side. Now the proxies handle this!
In combination with throttling, this approach enables high-frequency large-scale monitoring and allows for millions of checks per second without overloading the central Zabbix server. Proxies handle gigantic data volumes, with only a small part reaching the Zabbix server due to throttling, one to two orders of magnitude less.
Simpler low-level discovery
Let us remind you that low-level discovery (LLD) is a very powerful mechanism for automatically discovering any kind of resources for monitoring (file systems, processes, applications, services, etc.) and automatically creating data items, triggers, network nodes, and other objects based on them. This incredibly saves time, simplifies configuration, and allows the use of a single template for network nodes with different monitoring resources.
Low-level discovery required specially formatted JSON as input. Well, that will no longer be the case!
Zabbix 4.2 allows low-level discovery (LLD) to use arbitrarily formatted JSON data. Why is this important? It enables communication with external APIs, for instance, without resorting to scripts and utilizes the obtained information for the automatic creation of network nodes, data items, and triggers.
Combined with support for JavaScript, this creates fantastic opportunities for creating templates that work with various data sources, such as cloud APIs, application APIs, data in XML, CSV formats, and more.
Linking JSON with process information using LLD
The possibilities are truly limitless!
Support for TimescaleDB
What is TimescaleDB? It's standard PostgreSQL plus an extension module from the TimescaleDB team. TimescaleDB promises better performance through more efficient algorithms and data structures.
Additionally, another advantage of TimescaleDB is automatic table partitioning for historical data. TimescaleDB ensures speed and ease of maintenance! However, I must note that our team has not yet conducted a serious performance comparison with standard PostgreSQL.
At present, TimescaleDB is a fairly young and rapidly evolving product. Use with caution!
Easy tag management
Previously, tags could only be managed at the trigger level, but now tag management is much more flexible. Zabbix supports tags for templates and network nodes!
All detected issues receive tags not only from triggers but also from network nodes and the templates of those network nodes.
Defining tags for network nodes
More flexible auto-registration
Zabbix 4.2 allows filtering network nodes by name using regular expressions. This enables the creation of different discovery scenarios for various groups of network nodes. This is especially convenient if we are using complex naming rules for devices.
More flexible network discovery
Another improvement relates to the naming of network nodes. There is now the ability to manage device names during network discovery and obtain the device name from the metric value.
This functionality is very much needed, especially during network discovery using SNMP and the Zabbix agent.
Automatically assign the local network node name to the visible name
Checking the functionality of notification methods
Now you can send yourself a test message directly from the Web interface to check if the notification method is working. This functionality is especially useful for testing Zabbix integration scripts with various notification systems, task systems, and other external programs and APIs.
Remote monitoring of Zabbix infrastructure components
The ability to remotely monitor internal metrics of Zabbix servers and proxies (performance and operational metrics of Zabbix components) has been added.
Why is this needed? This functionality allows monitoring internal metrics of servers and proxies from the outside, enabling quick detection and notification of issues even if the components themselves are overloaded or, for example, if there is a large volume of unsent data on the proxy.
Support for HTML format in email messages
We are no longer limited to plain text and can create beautiful email messages thanks to the support for HTML format. It’s time to learn HTML + CSS!
Messages are easier to perceive even with minimal use of HTML
Access to external systems from network maps
Support for a whole new set of macros in user-defined URLs has been added for better integration of maps with external systems. This allows users to open, for example, a ticket in the task system with just one or two clicks on the node icon.
Open a ticket in Jira with one click
The discovery rule can be a dependent data element
Why is this necessary, you may ask? It allows using the main metric data for both detection and direct data collection. For instance, when collecting data from a Prometheus exporter, Zabbix will perform one HTTP request and immediately use the retrieved information for all dependent data elements: metric values and low-level discovery rules.
A new way to visualize problems on maps
Support for animated GIF images on maps has been added for more noticeable problem visualization.
Problematic devices have become more noticeable
Extracting data from HTTP headers in Web monitoring
The ability to select data from the received HTTP header has been added in Web monitoring.
This allows creating multi-step web monitoring scenarios or monitoring third-party APIs using an authorization token obtained in one of the steps.
Extracting AuthID from the HTTP header
Zabbix Sender uses all IP addresses
Zabbix Sender now sends data from all IP addresses listed in the ServerActive parameter of the agent's configuration file.
Convenient new filter in trigger configuration
The trigger configuration page has gained an enhanced filter for quick and convenient selection of triggers based on specified criteria.
Selecting triggers related to the K8S service
Displaying exact time
It's simple; now Zabbix shows the exact time when hovering the mouse over the graph.
Other new features
- A more predictable algorithm has been implemented for changing the order of widget placement in the dashboard.
- Ability for bulk changes of parameters for data element prototypes
- Support for IPv6 for DNS checks: 'net.dns' and 'new.dns.record'
- A 'skip' parameter has been added for 'vmware.eventlog' checks
- The preprocessing step execution error includes the step number
How to upgrade?
To transition from earlier versions, only the installation of (servers and proxies) and the new interface is necessary. Zabbix will automatically carry out the database update procedure. Installation of new agents is not required.
We conduct free webinars for those who want to learn more about Zabbix 4.2 and have the opportunity to ask questions to the Zabbix team.
Don't forget about the popular of the Zabbix community, where you can always get advice and answers to your questions in Russian from more experienced colleagues, and if you're lucky, from the Zabbix developers themselves. For newcomers, we recommend the .
Useful links
—
—
—
Source: habr.com
