In one of our , dedicated to planning infrastructure for the implementation of Zimbra Collaboration Suite in enterprises, it was mentioned that the main limitation when using this solution is the input-output speed of disk devices in mail storage systems. Indeed, when several hundred employees from a company access the same mail storage simultaneously, the bandwidth for reading and writing information from hard disks may not be sufficient for responsive service operation. While this may not be a significant issue for small Zimbra installations, in the case of large enterprises and SaaS providers, it can lead to unresponsive email services and, consequently, decreased employee efficiency as well as violations of SLA. Therefore, special attention should be paid to optimizing the performance of hard disks in mail storage when designing and operating large Zimbra installations. Let's consider two cases and try to understand what optimization methods can be applied to disk storage loads in each of them.
1. Optimization in Designing a Large-Scale Zimbra Installation
At the design stage of a high-load Zimbra installation, its administrator has to choose which storage system to use. To make this decision, it is essential to know that the primary load on the hard disks is generated by the database system MariaDB included in the Zimbra Collaboration Suite, the search system Apache Lucene, and the BLOB object storage. Therefore, high-speed and reliable equipment is necessary for the operation of these software products under high loads.
Under normal conditions, Zimbra can be installed on a RAID of hard disks or on storage connected via the NFS protocol. In very small installations, Zimbra can be installed on a regular SATA disk. However, in large installations, all these technologies exhibit various shortcomings, such as reduced writing speed or low reliability, which is unacceptable for large enterprises, let alone for SaaS providers.
That is why, in the context of large-scale Zimbra infrastructures, using SAN is the best option. It currently provides the highest throughput for storage devices and, due to the ability to connect a large amount of cache, its use carries virtually no significant risks for the enterprise. Using NVRAM, which is utilized in many SANs to speed up write operations, is a good idea. However, caching written data on the disks themselves should be disabled, as it can lead to irreparable damage to the media and data loss in case of power problems.
As for the choice of file system, using the standard Linux options Ext3/Ext4 is optimal. The main nuance related to the file system is that it should be mounted with the option -noatime. This option disables the recording of the last access time to files, significantly reducing read and write load. When creating an ext3 or ext4 file system for Zimbra, the following utility parameters should be used: mke2fs:
-j — To create the file system with an ext3/ext4 journal.
-L NAME — To create a volume label for later use in /etc/fstab.
-O dir_index — To use a hashed tree for faster file search in large directories.
-m 2 — To reserve 2% of the space in large file systems for the root directory.
-J size=400 — To create a large journal.
-b 4096 — To specify the block size in bytes.
-i 10240 — For message storage, this parameter should correspond to the average message size. It is important to pay close attention to this parameter, as it cannot be changed later.
In addition, it is recommended to enable dirsync for BLOB object storage, Lucene metadata search storage, and MTA queue storage. This is advised because Zimbra typically uses the utility fsync to guarantee the recording of the blob data to disk. However, when the Zimbra mail storage or MTA creates new files during message delivery, it becomes necessary to record the changes made to the corresponding folders on the disk. This is why even if the file has already been recorded to disk through fsync, the record of its addition to the directory may not be able to be written to disk in time and may consequently be lost due to a sudden server failure. Thanks to the use of dirsync these issues can be avoided.
2. Optimization with an Active Zimbra Infrastructure
It often happens that after several years of operating Zimbra, the number of its users increases significantly, and the service's responsiveness becomes less and less over time. The solution to this situation is obvious: new servers need to be added to the infrastructure so that the service runs as quickly as it used to. However, it is not always possible to immediately add new servers to the infrastructure to enhance its performance. IT managers often face prolonged approvals for purchasing new servers from accounting or the security department; additionally, suppliers can delay the delivery of the new server or provide the wrong one altogether.
Of course, it is best to build your Zimbra infrastructure with a margin to always have room for expansion and not depend on anyone. However, if a mistake has already been made, the IT manager can only minimize its consequences. For example, the IT manager can achieve a slight performance boost by temporarily disabling Linux system services that regularly access the hard drives and may negatively impact the Zimbra performance. Some services that can be temporarily turned off include:
autofs, netfs — Remote file system discovery services
cups — Print service
xinetd, vsftpd — Built-in *NIX services that you probably won't need
portmap, rpcsvcgssd, rpcgssd, rpcidmapd — Remote procedure call services, typically used in conjunction with network file systems
dovecot, cyrus-imapd, sendmail, exim, postfix, ldap — Duplicates of the core utilities included in the Zimbra Collaboration Suite
slocate/updatedb — Since Zimbra stores each message in a separate file, daily runs of the updatedb service may cause issues; therefore, it should be done manually during off-peak hours on the servers.
The savings in system resources from disabling these services may not be very significant, but even that can be quite useful in near emergency conditions. After a new server is added to the Zimbra infrastructure, it is recommended to re-enable the previously disabled services.
You can also optimize Zimbra's performance by moving the syslog service to a separate server, so it doesn't overload the hard drives of the mail storage during operation. Almost any computer will suffice for this purpose, including a cheap single-board computer like the Raspberry Pi.
Source: habr.com
