In previous posts, we shared instructions for setting up and based on Veeam. Today, we want to discuss backup solutions with Commvault. There won't be instructions, but we will explain what and how our clients are already backing up.
The storage system for backups based on Commvault in the OST-2 data center.
Initially, a check is performed: does the client device support power via PoE? A voltage of 2.8 to 10 volts is supplied, and the input resistance is determined. If the results obtained are satisfactory for powering via PoE, the power device proceeds to the next stage.
Commvault is a platform for backing up applications, databases, file systems, virtual machines, and physical servers. The source data can be located anywhere: on our side, on the client’s side, in another commercial data center, or in the cloud.
The client installs an agent on the backup objects – iData Agent – and configures it according to the required backup policies. The iData Agent collects the necessary data, compresses, deduplicates, encrypts, and transfers it to the DataLine backup system.
Proxy servers ensure the connectivity of the client network and our network, isolating the channels through which data is transmitted.
On the DataLine side, the data from the iData Agent is received by the Media Agent Server and sent for storage on the storage systems, tape libraries, etc. All of this is managed by CommServe. In our configuration, the main management server is located at the OST site, and the backup one is at the NORD site.
By default, client data is stored on one site, but it is possible to organize backups to two locations simultaneously or schedule the transfer of backups to the second site. This option is called 'auxiliary copy of data.' For example, all full backups at the end of the month will automatically be duplicated or moved to the second site.
The operational scheme of the Commvault backup system.
The backup system primarily operates on VMware virtualization: CommServe, Media Agent, and Proxy servers are deployed on virtual machines. If the client uses our equipment, backups are placed on Huawei OceanStor 5500 V3 storage systems. For backing up client storage systems and retaining backups on tape libraries, separate Media Agents on physical servers are used.
What matters to clients?
From our experience, clients who choose Commvault for backups pay attention to the following points.
Console. Clients want to manage backups independently. All basic operations are available in the Commvault console:
- adding and removing servers for backup;
- configuring iData Agent;
- creating and manually initiating jobs;
- self-restoration of backups;
- setting up notifications for the status of backup tasks;
- restricting access to the console based on user roles and groups.
Deduplication. Deduplication helps to find and remove duplicate data blocks during the backup process. As a result, it helps save storage space and reduces the volume of data transmitted, lowering bandwidth requirements. Without deduplication, backups would occupy two to three times the volume of the original data.
In the case of Commvault, deduplication can be configured on the client side or on the Media Agent side. In the first case, non-unique data blocks will not even be transmitted to the Media Agent Server. In the second case, the duplicate block is discarded and not written to the storage system.
This block-level deduplication is based on hash functions. Each block is assigned a hash that is stored in a hash table, a kind of database (Deduplication Database, DDB). During data transmission, the hash is checked against this database. If such a hash already exists in the database, the block is marked as non-unique and is not transmitted to the Media Agent Server (in the first case) or not written to the storage system (in the second).
Thanks to deduplication, we manage to save up to 78% of storage space. Currently, 166.4 TB is stored on the storage system. Without deduplication, we would have to store 744 TB.
Ability to restrict permissions. In Commvault, there is an option to set different access levels for managing backups. The so-called 'roles' define what actions a user can take regarding backup objects. For example, developers may only restore a database server to a specific location, while an administrator can initiate an on-demand backup for the same server and add new users.
Encryption. Data can be encrypted during backup through Commvault in the following ways:
- On the client agent side, the data will be transferred to the backup system already in an encrypted format.
- On the Media Agent side;
- At the channel level, the data is encrypted on the client agent side and decrypted on the Media Agent Server.
Available encryption algorithms include: Blowfish, GOST, Serpent, Twofish, 3-DES, AES (Commvault recommended).
A bit of statistics
By mid-December, we have 27 clients backed up using Commvault. Most of them are retailers and financial organizations. The total volume of original backup data is 65 TB.
About 4,400 jobs are performed daily. Below is the statistics for completed jobs over the past 16 days.
The most backed up through Commvault are the Windows File System, SQL Server, and Exchange databases.
And now the promised case studies. Although anonymized (NDA says hello :)), they provide an insight into how and why clients use backup based on Commvault. Below are the cases from clients using a unified backup system, i.e., shared software, Media Agent Servers, and storage systems.
Case 1
Customer. A Russian trading and manufacturing company in the confectionery market with a distributed network of branches across Russia.
Task.Backup organization for Microsoft SQL databases, file servers, application servers, and Exchange Online mailboxes.
Original data is located in offices across Russia (more than 10 cities). Backup is needed at the DataLine site with subsequent data recovery at any of the company's offices.
At the same time, the client wanted complete self-management with access restrictions.
Retention period – one year. For Exchange Online – 3 months for operational copies and one year for archives.
Solution. An additional copy was set up for the databases at a second site: the last complete backup of the month is transferred to another site and stored there for a year.
The quality of channels from the client's remote offices did not always allow making backups and restorations in optimal time. To reduce the volume of transmitted traffic, deduplication was set up on the client side. This made the time for complete backup acceptable considering the remoteness of the offices. For example, a full backup of a 131 GB database from St. Petersburg takes 16 minutes. From Yekaterinburg, a 340 GB database is backed up in 1 hour and 45 minutes.
Using roles, the client configured different permissions for their developers: either for backup only or restoration.
Case 2
Customer. A Russian chain of children's goods stores.
Task. Backup organization for:
a highly loaded MS SQL cluster based on 4 physical servers;
virtual machines with websites, application servers, 1C, Exchange, and file servers.
All of the client's specified infrastructure is distributed between the OST and NORD sites.
RPO for SQL servers is 30 minutes, for others – 1 day.
Storage depth is from 2 weeks to 30 days depending on the type of data.
Solution. They chose a combination of solutions based on Veeam and Commvault. Veeam is used for file backups from our cloud. Database servers, Active Directory, mail and physical servers are backed up through Commvault.
To achieve high backup speeds, the client allocated a separate network adapter for backup tasks on physical servers with MS SQL. A full backup of a 3.4 TB database takes 2 hours and 20 minutes, while full restoration takes 5 hours and 5 minutes.
The client had a large volume of source data (almost 18 TB). If the data were to be stored on a tape library, as the client had previously done, it would require several dozen cartridges. This would complicate the management of the client's entire backup system. Therefore, in the final implementation, the tape library was replaced with a storage area network (SAN).
Case 3
Customer. A chain of supermarkets in the CIS.
Task. The customer wanted to organize backups and restorations of SAP systems hosted in our cloud. For SAP HANA databases, RPO=15 minutes, and for virtual machines with application servers, RPO=24 hours. Storage depth is 30 days. In case of a failure, RTO=1 hour, and for restoration upon request, RTO=4 hours.
Solution. For HANA databases, backup of DATA files and log files was configured with a specified frequency. Log files were archived every 15 minutes or when a certain size was reached.
To reduce database recovery time, we configured a two-tier backup storage based on SAN and tape library. Operational copies are stored on disks with the ability to restore at any moment within a week. When a backup becomes older than 1 week, it is moved to the archive, to the tape library, where it is kept for an additional 30 days.
A complete backup of one of the databases with a size of 181 GB takes 1 hour and 54 minutes.
When setting up backup, the SAP backint interface was used, allowing integration of third-party backup systems with SAP HANA Studio. Therefore, backups can be managed directly from the SAP console. This simplifies life for SAP administrators, who do not need to get used to a new interface.
Backup management is also available to the client through the standard Commvault client console.
That's all for today. Please ask questions in the comments.
Source: habr.com
