🥇What's new in Zabbix 5.0

In mid-May, Zabbix 5.0 was released, and we organized a series of online meetups in different languages in order to visually demonstrate all the changes and innovations to the community. We invite you to read the report of the Executive Director and creator of Zabbix Alexey Vladyshev, in which he told step by step what's new in Zabbix 5.0.

Zabbix 4.2 and Zabbix 4.4

Let's start with the changes introduced in Zabbix 4.0 due to the use of LTS versions.
In the version of Zabbix 4.2, which was released in April 2019, the following features appeared:

High-frequency throttling monitoring that scales and boosts NVPS, which means faster problem detection and alerting without heavy load on Zabbix.
Collecting data using the HTTP agent.
Data collection support from Prometheus Pro.
Preprocessing supports validation and JavaScript, which allows you to transform any collected data.
Pre-processing on the proxy side, which allows for more efficient scaling using proxies.
Improved tag management - meta-information at the level of events and issues that is convenient to work with, because tags are supported at both the template level and the host level.

In September last year, Zabbix 4.4 was released, which offered the following features:

New Zabbix agent.
Webhook support for alerts and notifications, allowing integration with external systems.
TimescaleDB support.
The built-in knowledge base for metrics and triggers has become visible to Zabbix users. For example, users can use the description of items and triggers in Monitoring > Latest data.
The new standard for templates.

Zabbix 5.0

Today we will talk about the LTS release of Zabbix 5.0, which will be supported for 5 years. Support for version 4.4 ends after one month. The LTS release of Zabbix 3.0 will be supported for another 3,5 years.

Zabbix provides monitoring of many things, the list of which can be specified on the page http://www.zabbix.com/integrations, where monitoring templates and plugins are presented, including for the new agent.

Available templates for monitoring and integration

In addition, there are integration options with various systems, including ticket systems, ITSM systems, and Webhook messaging systems.

Integration options

Zabbix 5.0 has extended built-in support for integration with various ticket systems, as well as notification systems:

Integration with various systems

The list of built-in templates for monitoring applications and devices has been expanded:

Built-in templates for application and device monitoring

All updates are available for download at Git repository.

Any user or developer can participate in Zabbix with ready-made developments - templates or plugins, using a simple procedure:

Signing Zabbix Contributory Agreement (ZCA) on https://www.zabbix.com/developers.
Placement of Pull Request on https://git.zabbix.com.
Consideration of the application by the development team. If a plugin or template complies with the Zabbix standards, it is included in the product and the work of such a developer will be officially supported by the Zabbix team.

Zabbix is open source software available for viewing, studying and modifying. The user is given the opportunity to freely use the product, participate in the improvement of the program or use the code for their new programs. On the other hand, the Zabbix team does its best to make Zabbix easy to install on various platforms.

Zabbix developers offer packages for almost all the most popular distributions and various virtualization platforms. In addition, Zabbix can be installed in the public cloud with a single click. Zabbix is also available on the Red Hat Openshift or OpenStack platforms.

Zabbix packages for distributions and platforms

Support for Zabbix Agent 2 for Windows and Linux

The new Zabbix Agent 2 is one of the best solutions on the market.

Offers a plugin-based framework and supports data collection scripts that can run for hours.
Supports parallel active checks, constant connections to external systems, which is useful, for example, for effective database monitoring.
Supports hooks and events, which is important for monitoring, for example, MQTT devices.
The new version of the agent is easy to install (because the new agent supports all the previous functionality).

In addition, support for persistent data storage is offered for the new agent in Zabbix 5.0. Previously, unsent information was stored only in the agent's buffer memory, but in the new version, it is possible to configure the storage of such information on disk.

Persistent data storage

This is important in case of monitoring critical systems and unstable communications, since a large amount of critical data is saved before being sent to the Zabbix server. The option is also useful for satellite connections that may not be available for a long time.
IMPORTANT! Zabbix 5.0 retains support for Zabbix Agent 1.

Security changes in Zabbix 5.0

1. The new version supports an HTTP proxy for webhook, which allows you to connect from Zabbix server to external notification systems in a more secure and manageable way.

If you need to integrate a Zabbix server on the local network with an external system, such as JIRA in the cloud, you can maintain a connection through an HTTP proxy, which improves the controllability and reliability of the connection.

2. Both for the old and for the new agent, it is possible to choose which checks should be available on a particular agent. For example, you can limit the number of checks by actually creating white and black lists, define supported keys.

Whitelist for MySQL related checks
```
AllowKey=mysql[*] 
DenyKey=*
```
Blacklist to deny all shell scripts
```
DenyKey=system.run[*]
```
Blacklist to deny access to /etc/password
```
DenyKey=vfs.file.contents[/etc/passwd,*]
```

3. You can select encryption algorithms for all Zabbix components to avoid using insecure ciphers for TLS connections. This is important for monitoring environments for which certain security standards apply.

Selecting encryption algorithms for TLS connections

4. Zabbix 5.0 added support for encrypted database connections. Currently only encrypted connection with PostgreSQL and MySQL is available.

Encrypted database connections

5. Zabbix 5.0 switched from MD5 to SHA256 for storing user password hashes in the database as it is currently the most secure algorithm.

6. Zabbix 5.0 supports secret user macros to store any sensitive information such as passwords and API tokens that end users do not have access to.

Secret Macros

7. All Zabbix connections to external systems and internal connections to agents are secure. Encryption is supported using TLS certificates, or using pre-shared key encryption to connect to agents and proxies, or HTTPS. Security on the agent's side can be enhanced with whitelists and blacklists. The interface works over HTTPS.

Secure connections

8. Support for SAML to provide a single point of authentication with a trusted identity provider so user credentials don't leave the firewall.

SAML authentication

SAML support allows Zabbix to be integrated with various on-premises and cloud identity providers such as Microsoft ADFS, OpenAM, SecurAuth, Okta, Auth0, as well as Azure, AWS or Google Cloud Platform.

Usability of Zabbix 5.0

1. User interface optimized for wide screens. We have moved the menu from the top, where there is always not enough space, to the left side of the screen. The menu is still displayed in full, minimal, and hidden mode.

Interface optimized for the wide screen

2. Copy widgets from panels allows you to create new PANELS very quickly. To do this, select the desired widget in the PANEL, click Copy

Widget Copy

and insert the widget into the desired panel.

Paste the copied widget

3. Export charts. To copy the graph and send it, for example, by e-mail, you can get the graph in PNG format by selecting the desired widget and clicking download image.

Graph export

4. Filtering by tags: Problem by severity and Problem hosts. It became possible, for example, to collect data on all problems associated with one network node in one data center.

Filtering by tags

5. Support for modules to extend the Zabbix interface. To install an independent module, you need to copy it to a specific directory. Modules allow you to expand the existing interface functionality, create new pages, change the menu structure, for example, add items.

Any user can write and integrate a module. To do this, the module is copied to the modules folder, after which it becomes visible for the interface, where it can be turned on and off.

Adding a new module

6. Ease of navigation through resources associated with network nodes. In Monitoring > Hosts displays a list of devices monitored by Zabbix: hosts, services, network devices, etc. In addition, quick navigation to screens, graphs, and device-specific issues is available.

We removed the tabs Monitoring > Graphs and Monitoring > Webs, and all navigation is via Monitoring > Hosts. The displayed information can be filtered, including by tags, which allows you to display disabled devices

Navigating the resources associated with hosts

For example, you can select devices that belong to end user services by selecting 'Service', as well as setting the level of importance of these problems.

Filtering options

7. New preprocessing operation - 'Replacement' allows you to do some useful things that previously could only be done with regular expressions, which is quite complicated for many users.
Replace allows you to actually replace one string or character with another, which allows you to simply convert the data received in text format into a numeric representation.

Replace statement

8. JSONPath Statement, which allows you to extract attribute names in a convenient form

Operator for JSONPath

9. Display Zabbix email messages. In previous versions, all email messages from Zabbix in the folder Inbox displayed in a list. Starting with Zabbix 5.0, messages will be grouped by issue.

Grouping email messages from Zabbix

10. Support for custom macros for IPMI for username and password. If secret macros are used for the username and password, access to their value will be denied.

Support for custom macros

11. Mass change of user macros for hosts. In the new version, you can open a list of templates, select a list of hosts and add macros or change the values of existing macros,

Adding and editing custom macros

and also remove some specific or all macros from selected templates for hosts.

Deleting individual or all custom macros

12. Message format control at the notification method level. In media types tab appeared media templates with message templates.

Notification Method Templates

You can define different templates for different message types.

Defining a template for a message type

In previous versions, you had to manage these messages at the action level, defining messages and a default subject.

Action level template management

In the new version, everything can be defined at the global level, and at the message level, global settings can be rewritten.

Template management at the global level

For most users, defining template formats at the media level is sufficient. Moreover, after importing some new notification method, all relevant template formats are already part of it.

13. Wider use of JavaScript. JavaScript is used for preprocessing scripts, for Webhooks, etc. On the command line, working with JavaScript is not easy.
Zabbix 5.0 uses a new utility − zabbix_js, which runs JavaScript that takes in data, processes it, and generates output values.

zabbix_js utility

Examples of using the zabbix_js utility

14. Support for text operations with trigger expressions allows you to check the versions of installed components, compare values with any constants, while a custom macro can be a constant,

{host:zabbix.version.last()}="5.0.0"
{host:zabbix.version.last()}="{$ZABBIX.VERSION}

compare the last value with the previous one, for example, when it comes to text data,

{host:text.last()}<>{host.text.prev()}

{host:text.last(#1)}<>{host.text.prev(#2)}

or compare text values of different metrics.

{hostA:textA.last()}={hostB:textB.last()}

15. Automation and discovery.

New JMX checks are available to get and discover a list of JMX counters, which is very useful, for example, for monitoring Java applications, as well as automating the creation of monitoring elements, metrics, triggers, and graphs.
```
jmx.get[]
```
и
```
jmx.discovery[]
```
JMX checks
The new version has a key for monitoring Windows performance counters, which is supported by the old and new agents in Russian and English and allows, for example, to detect the number of processors, file systems, services, etc.

Monitoring Windows performance counters with a key perf_counter
ODBC monitoring just got a whole lot easier. Previously, all parameters for ODBC monitoring had to be described in an external file /etc/odbc.ini, which was not accessible from the Zabbix interface. In the new version, almost all parameters can be part of the metric key.

Metric key with parameter description

In the new version, you can set the server name and port at the metric level, and the access name and password using secret macros for security.

Using secret macros
When using the IPMI protocol for equipment monitoring, it became possible to create simpler templates for automation using ipmi.get.

ipmi.get

16. Testing data elements from the interface. Zabbix 5.0 introduced the ability to test some items and, more importantly, item templates from the interface.

Item Testing

Any problems that arise are displayed in the interface.

Display problems in the interface

A similar algorithm is used for item templates. Also, if an item is not supported, you can simply click Test.

17. Testing notification methods, which was introduced in Zabbix 4.4, is preserved, which is important when integrating Zabbix with other systems, such as ticket systems.

Testing notification methods

18. Support for custom macros for item prototypes. You can use LLD macros to define values for custom macros.

Using LLD Macros to Define Custom Macro Values

19. Support Float64 data, which are needed mainly for monitoring very large values, is required in Zabbix to support data received from Prometheus agents.
If Zabbix 5.0 is installed, data will not be automatically migrated to the Float64 standard. The user still has the option to use the old data types. Float64 migration scripts are run manually and change data types in historical tables. Automatic replacement is not used because it takes a very long time.

20. Zabbix 5.0 scalability improvements: interface optimization and elimination of "bottlenecks"

Eliminated drop-down lists, for example, for selecting hosts, because this feature is not scalable.
There are "built-in" limits for table sizes Overview.
There are new opportunities in Monitoring > Hosts > Graphs.
The paging function appeared (Monitoring > Hosts > Web) where it was not.

21. Improved compression
Compression in Zabbix is based on the PostgreSQL extension TimescaleDB (since Zabbix 4.4). TimescaleDB provides automatic database partitioning and improves database performance because TimescaleDB performance is virtually independent of database size.

In Zabbix 5.0 Administration > General > Housekeeping you can configure, for example, compression of data older than 7 days. This significantly reduces the required disk space (nearly ten times, according to users), which improves disk space savings and improves performance.

Compression with TimescaleDB

22. Configure SNMP at the interface level. In Zabbix 5.0, instead of three types of items, only one is used - SNMP agent. All SNMP attributes have been moved to the host interface layer to simplify templates, switch between SNMP versions, etc.

Configuring SNMP at the interface level

23. Dependence of monitoring the availability of network nodes on the availability of proxies allows you to display the problem of proxy availability as a priority in case of unavailability of hosts when monitoring using a trigger with the function notation:

{HostA:item.nodata(1m)}=1

The availability of hosts is determined by the availability of the proxy

Function notation by default takes into account the availability of the proxy. For a more rigorous check that does not take into account the availability of the proxy, the second parameter is used - strict:

{HostA:item.nodata(1m,strict)}=1

24. Management of low-level discovery rules. Zabbix 5.0 introduced an LLD filter that allows you to view, among other things, unsupported discovery rules

LLD filter

25. Ability to unacknowledge a problem (unacknowledge) allows you to fix errors and is useful when creating workflows that depend on problem confirmation.

Removing problem confirmation

26. Changing the rules of low-level discovery - the ability to add exceptions when detecting objects as a result of monitoring file systems, which allows you to create or not create certain objects, triggers, data elements, etc., at low-level detection, change the severity of problems, add tags for certain objects, exclude objects, for example, temporary file systems, from search, change data update interval, etc.

Exception from low-level detection of temporary file systems

For example, you can change the priority level of triggers for discovered Oracle file systems while leaving the priority level of triggers for other file systems at the same level.

Change the priority level of triggers for individual file systems

27. New macros in Zabbix 5.0 improve the quality of monitoring.

New macros in Zabbix 5.0

28. Other innovations in Zabbix 5.0:

Improvements in Zabbix 5.0

29. End of support

Unsupported functionality

Conclusion

Upgrading to Zabbix 5.0 is very easy! Install and run the new server binaries and frontend files and the server will automatically update your database.
Information about the Zabbix upgrade procedure is available at:
https://www.zabbix.com/documentation/current/manual/installation/upgrade_notes_500

IMPORTANT!

Updating historical data to Float64 format is optional.
TimescaleDB data is read-only.
The minimum required version is PHP7.2.
DB2 is not supported as backend for Zabbix server

(!) Videos and slides of presentations by Alexey Vladyshev and other speakers of Zabbix Meetup Online (Russian) can be viewed here.

Source: habr.com

What's new in Zabbix 5.0