ProHoster > Blog > Administration > Upgrading Check Point from R77.30 to 80.20

Upgrading Check Point from R77.30 to 80.20

Upgrading Check Point from R77.30 to 80.20

In the fall of 2019, Check Point stopped supporting R77.XX versions, and it was necessary to upgrade. A lot has already been said about the difference between versions, the pros and cons of switching to the R80. Let's better talk about how to actually update Check Point virtual appliances (CloudGuard for VMware ESXi, Hyper-V, KVM Gateway NGTP) and what can go wrong.

So, we had 2 CCSE engineers, more than a dozen Check Point R77.30 virtual clusters, several clouds, a few hotfixes and a whole sea of ​​\uXNUMXb\uXNUMXbvarious bugs, glitches and all that, of all colors and sizes, and also very tight deadlines. Let's go!

Contents:

Prepare
Updating the management server
Updating the cluster

Upgrading Check Point from R77.30 to 80.20

This is what a typical client cloud infrastructure looks like with a virtual Check Point

Prepare

The first step is to check if there are enough resources for the update. The recommended minimum requirements for R80.20 now look like this:

Device

CPU

RAM

HDD

security gateway

2 core

4 Gb

From 15 GB

SMS

2 core

6 Gb

β€”

Recommendations are described in the document CP_R80.20_GA_Release_Notes.

But we will be realistic. If this is enough in the most minimal configuration, then, as practice shows, we usually have https inspection enabled, SmartEvent works on SMS, etc., which, of course, requires completely different capacities. But in general, not more than for R77.30.

But there are nuances. And they concern, first of all, the size of physical memory. Many operations directly during the update process will require hard disk space.

For the management server, the amount of free space on the disk will very much depend on the size of the current logs (if we want to keep them) and on the number of Database Revisions saved, although we will not need them in large numbers. Of course, for cluster nodes (unless you also store logs locally) this does not matter. Here's how you can check for the required space:

  1. We connect to the Smart Management Server via ssh, go into expert mode and enter the command:

    [Expert@cp-sms:0]# df -h

  2. At the output, we will see something like this configuration:

    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/vg_splat-lv_current 30G 7.4G 21G 27% /
    /dev/sda1 289M 24M 251M 9% /boot
    tmpfs 2.0G 0 2.0G 0% /dev/shm
    /dev/mapper/vg_splat-lv_log 243G 177G 53G 78% /var/log

  3. We are currently interested in the section / var / log

Please note that depending on the policy for storing and deleting old log files, as well as the size of the exported database, more space may be required. If during the creation of the archive there is less free space than specified in the log file storage policy, the system will start deleting old logs and will NOT include them in the archive.

Also, for the update process itself, the system will need at least 13 GB of unallocated hard disk space. You can check its presence with the command:

[Expert@cp-sms:0]# pvs

We will see something like this output:

PV VG Fmt Attr PSize PFree
/dev/sda3 vg_splat lvm2 a- 141.69G 43.69G

In this case, we have 43 GB. There are enough resources. You can start updating.

Upgrading the Check Point SMS Management Server

Before starting work, do the following:

  1. Install the Migration Tools package on the management server. To do this, you need to upload the image from the portal Check Point.
  2. Upload the archive to the management server via WinSCP to the folder /var/log/UpgradeR77.30_R80.20 (if necessary, create a folder first).
  3. Connect to the management server via SSH and go to the folder with the archive:cd /var/log/UpgradeR77.30_R80.20/
  4. Let's unzip the file:tar -zxvf ./<filename>.tgz
  5. Run the pre_upgrade_verifier utility with the command: ./pre_upgrade_verifier -p $FWDIR -c R77 -t R80.20
  6. Upon execution of the command, a report on incompatible settings will be generated. It is available at: /opt/CPsuite-R77/fw1/log/pre_upgrade_verification_report.(xls, html, txt). It is more convenient to upload it through SCP and watch it through the browser.
    To remove all incompatible settings, use SK117237.
  7. Then re-run the pre_upgrade_verifier utility to make sure that all causes of incompatibility are eliminated.
  8. Next, we collect information about network interfaces, the routing table and upload the GAIA configuration:
    ip a > /var/log/UpgradeR77.30_R80.20/cp-sms-config.txt
    ip r > /var/log/UpgradeR77.30_R80.20/cp-sms-config.txt
    clish -c "show configuration" > /var/log/UpgradeR77.30_R80.20/cp-sms-config.txt
  9. We upload the received file through SCP.
  10. We make a snapshot at the virtualization level.
  11. Increase SSH session timeout to 8 hours. Here you are lucky: depending on the size of the exported database, it can last from several minutes to several hours. For this: 
    [Expert@HostName]# clish -c "show inactivity-timeout" watch the current timeout clish,

    [Expert@HostName]# clish -c "set inactivity-timeout 720" specify a new timeout clish (in minutes),

    [Expert@HostName]# echo $TMOUT look at the current timeout expert mode,

    [Expert@HostName]# export TMOUT=3600 specify a new timeout expert mode (in seconds), if you set the value to 0, then timeout will be turned off.

  12. We load and mount the SMS.iso installation image to the virtual machine.

    Before the next step, ALWAYS double-check that you have enough unallocated space on your hard drive (I remind you, you need 13 GB). 

  13. Before starting the export of the configuration, we change the log file with the command: fw logswitch

Export configuration and logs

  1. Run the migrate_export utility to upload the configuration. To do this, go to the previously created folder: cd /var/log/UpgradeR77.30_R80.20/ and use the command: ./migrate export -l /var/log/UpgradeR77.30_R80.20/SMS_w_logs_export_r77_r80.tgz

    or

    go to the folder: cd $FWDIR/bin/upgrade_tools/ ΠΈ
    run the command from there: ./migrate export -l /var/log/UpgradeR77.30_R80.20/SMS_w_logs_export_r77_r80.tgz

  2. Remove checksum from the archive: md5sum /var/log/UpgradeR77.30_R80.20/SMS_w_logs_export_r77_r80.tgz
  3. We save the resulting value in a notepad.
  4. We connect to SMS via SCP and upload the archive with the configuration to the workstation. Be sure to use file transfer in Binary format.

Export SmartEvent Database

Here we need a pre-installed SMS version R80. Any test will do. 

  1. From SMS we need a script located here:$RTDIR/bin/eva_db_backup.csh
  2. Loading the script via SCP eva_db_backup.csh to folder: /var/log/UpgradeR77.30_R80.20/
  3. Connect via SSH to SMS. Copy file to folder: cp /var/log/UpgradeR77.30_R80.20/eva_db_backup.csh
    $RTDIR/bin/eva_db_backup.csh
  4. Change encoding: dos2unix $RTDIR/bin/eva_db_backup.csh
  5. Adding an owner: chown -v admin:root $RTDIR/bin/eva_db_backup.csh
  6. Adding rights: chmod -v 0755 $RTDIR/bin/eva_db_backup.csh
  7. We start the export of the SmartEvent database: $RTDIR/bin/eva_db_backup.csh
  8. Uploading the received files via SCP: $RTDIR/bin/<date>-db-backup.backup ΠΈ $RTDIR/bin/eventiaUpgrade.tar to the workstation.

Update

  1. We pass to WebUI GAIA SMS β†’ CPUSE β†’ Show all packages.
  2. In the case when CPUSE gives an error connecting to the Check Point cloud, we check the DGW, DNS and Proxy settings.
  3. If everything is correct, but the error does not disappear, then you need to update the CPUSE manually, guided by sk92449.
  4. Download the image and go verifier. If necessary, we eliminate inconsistencies.

    As a result, you should see the following message:

    Upgrading Check Point from R77.30 to 80.20

  5. Choosing R80.20 Fresh Install and Upgrade for Security Management.
  6. When installing the update, select Clean Install. After installation, the system will reboot.
  7. Passing First Time Wizard.
  8. After gaining access, we check the accounts.
  9. We connect to SMS via SSH and change our user's shell to /bin/bash/:

    set user <username> shell /bin/bash/

    saveconfig (in case we want to leave bin/bash/ as the default shell even after a reboot).

  10. Next, we connect to SMS via SCP and in Binary mode we transfer the archive with the configuration SMS_w_logs_export_r77_r80.tgz to folder /var/log/UpgradeR77.30_R80.20/
  11. Remove checksum from the archive: md5sum /var/log/UpgradeR77.30_R80.20/SMS_w_logs_export_r77_r80.tgz and compare with the previous value. Checksum must match.
  12. Increase SSH session timeout to 8 hours. For this:

    [Expert@HostName]# clish -c "show inactivity-timeout" watch the current timeout clish,

    [Expert@HostName]# clish -c "set inactivity-timeout 720" specify a new timeout clish (in minutes),

    [Expert@HostName]# echo $TMOUT look at the current timeout expert mode,

    [Expert@HostName]# export TMOUT=3600 specify the new timeout expert mode (in seconds). If set to 0, then timeout will be disabled.

  13. To import settings, run the migrate import utility. To do this, go to the folder: cd $FWDIR/bin/upgrade_tools/and run the import: ./migrate imp
    ort -l /var/log/UpgradeR77.30_R80.20/SMS_w_logs_export_r77_r80.tgz

Enjoy life for the next couple of hours. DO NOT DISCONNECT SSH SESSION during the procedure. At the end of the migrate process, either a success message or an error will be returned. 

Post-update checklist

  1. Availability of resources.
  2. SIC with GW.
  3. Licenses. If licenses are displayed incorrectly or are not displayed on SMS, run the command vsec_central_licence to distribute licenses.
  4. Setting the policy. 

Importing a SmartEvent Base

  1. Activate the SmartEvent blade.
  2. We connect via WinSCP to SMS and transfer the previously uploaded files in binary mode <date>-db-backup.backup ΠΈ eventiaUpgrade.tar to folder /var/log/UpgradeR77.30_R80.20/
  3. Run the script with the command: $RTDIR/bin/eventiaUpgrade.sh -upgrade /var/log/UpgradeR77.30_R80.20/eventiaUpgrade.tar
  4. Checking the status: watch -n 10 eventiaUpgrade.sh
  5. Checking the logs in SmartEvent. DREAM!

Upgrading the Check Point GW Cluster (Active/Backup)

Before starting work

  1. We save the GAIA configuration from each node of the cluster to a file, for this use the command: clish -c "show configuration" > ./<Filename>.txt
  2. Uploading files using WinSCP.
  3. We connect to the WebUI of both nodes and go to the tab CPUSE β†’ Show all packages.
  4. Find the update package for the version R80.20 Fresh installpush Download.
  5. We check that the CCP protocol is working in the mode On the roadcast, for this we enter the command: cphaprob -a if
    If the mode is selected Multicast, change it with the command: cphaconf set_ccp broadcast (the command is executed on each node).
  6. Set Downtime for the involved nodes in your monitoring system.
  7. Verify that options are enabled at the virtualization level MAC Address Change ΠΈ Forged Transmits for sync network.

Update

  1. We connect via ssh to the Active node and run the command to monitor the state of the cluster: watch -n 2 cphaprob stat
  2. We return to the WebUI Stanby nodes to the tab CPUSE and for the selected package R80.20 Fresh install launch verifier.
  3. Analyzing the Verifier report. If the installation is allowed, go ahead.
  4. Choose a package R80.20 Fresh install and run Upgrade. During the Upgrade process, the system will reboot. GAIA settings are saved. At the time of reboot, we monitor the state of the cluster. After uploading, the status of the updated node should change to READY. In some cases, we had a moment when a node that had not yet been updated would switch to the Active Attention status and stop displaying the status of the updated node. Don't be afraid - this option is also acceptable.
  5. After the update is completed, open smart dashboard.
  6. Open the cluster object and change the cluster version from R77.30 to R80.20. Click OK. If an error occurs when saving changes:
    An internal error has occurred. (Code: 0x8003001D, Could not access file for write operation),
    follow SK119973. After that, save the changes and click Install Policy.
  7. In the settings, uncheck the option For gateway clusters, if installation on a cluster member fails, do not install on that cluster.
  8. We set the policy. The system will generate an error for an Active node that has not been updated yet.
  9. We connect to the updated node via ssh and run the command to monitor the state of the cluster: watch -n 2 cphaprob stat
  10. We connect to the WebUI Active node and go to the tab CPUSE β†’ Show all packages.Find the update package for the version R80.20 Fresh install, we press Download.
  11. Set Downtime for the involved nodes in your monitoring system.
  12. We return to the WebUI Active nodes to the tab CPUSE and for the selected package R80.20 Fresh install launch verifier.
  13. Analyzing the Verifier report. If the installation is allowed, go ahead.
  14. Choose a package R80.20 Fresh install and run upgrade. During the Upgrade process, the system will reboot. GAIA settings are saved. At the time of the reboot, we monitor the state of the cluster on the already updated node. After a reboot, the cluster state on the upgraded node will change from READY to ACTIVE.
  15. When the Upgrade process is completed, launch SmartDashboard and install the policy.

Post-update checklist

  • Event logs in SmartLog, status of VPN tunnels.
  • GAIA settings.
  • Restoring a cluster after a test Failover.
  • Licenses and contracts. If the licenses are displayed incorrectly or are not displayed on SMS, run the command. vsec_central_licence for license distribution.
  • CoreXL.
  • SecureXL.
  • Hotfix and CPinfo on two nodes.

Conclusion

In general, at this point, everything - you have updated.

For us, the whole process took an average of 6 to 12 hours, depending on the size of the exported bases. The work was carried out for two nights: one for updating SMS, the second for the cluster.

There was no traffic downtime, despite the fact that we tested all the above errors ourselves.

Of course, sometimes completely new difficulties can arise during the upgrade process, but this is Check Point, and as we all know, there is always a hotfix!

Good luck with your black and pink nights and updates!

Source: habr.com

Add a comment