Checkmk
DeutschEspañolFrançaisItaliano
master2.2.02.1.02.0.01.6.0
to checkmk.com

1. Introduction

There are a number of recommended methods for upgrading the Linux distribution release on a Checkmk server. Which one is the right one for you depends on the capabilities of your IT landscape, the inventory of data on the Checkmk installation, and the anticipated downtime. We present the various methods with their respective requirements here.

Essentially, one method is to update on the spot, sometimes referred to as in situ (from Latin). The other method uses a backup as an archive, on which the reinstallation of the basic system and finally the restoration of the same archive takes place, sometimes correspondingly known as ex situ. In English the analogous terms in-place, or out-of-place are sometimes used for these two methods.

Combinations of the methods presented here are also possible, for example, if you have placed /opt/omd or /opt/omd/sites on a separate mount point.

We will demonstrate the release upgrade here using the change from Ubuntu 20.04 (Focal) to 22.04 (Jammy) as an example. For other distributions, the upgrade and package management commands may differ from this to a greater or lesser extent.

Our example system uses only a single site. If you run multiple sites on the system to be upgraded (for example, one for production and one for testing), repeat the appropriate site-specific commands for each.

If you want or need to combine a release upgrade of the Linux distribution with a version update of Checkmk (for example because an older Checkmk version is no longer offered for a newer Linux distribution version), use our update matrix to determine the suitable order to be performed.

2. Some notes on backups

We IT professionals do not really need to explain the importance of a sufficiently comprehensive data backup, which is also checked for consistency. In the ideal scenario — for example, when running Checkmk in a virtual machine — a full backup is easily achieved by copying hard disk images.

As a minimum, we recommend:

  • Creating a backup using the tools provided by Checkmk, either via the command line using the omd command or via the Setup. This can be done in preparation while the system is still running.

  • A backup of the contents of /opt/omd/sites using operating system tools after stopping the site and unmounting the site-specific /tmp file system. This can be done during the upgrade.

Keep the backup separate from the machine being upgraded and test the backup before making the first changes to the production system.

If you chose a copy-on-write file system such as ZFS or BTRFS when you installed the Checkmk server, you should take snapshots before upgrading. These do not replace a separately stored backup, but can help significantly reduce downtime when restoring the system to its original condition if the upgrade fails.

3. Upgrading in-place

This method is often useful when using Checkmk on dedicated hardware with large data sets, where copying an archive back and forth to restore it would significantly increase downtime.

3.1. Preparation

The preparation consists mainly of removing all Checkmk packages that are already redundant. This way you can avoid package management problems during the actual upgrade. In addition, you should have already downloaded the correct Checkmk installation package for the new distribution version.

  1. First get an overview of all currently installed Checkmk versions…​

    root@linux# omd versions
2.2.0p25.cre
2.3.0p1.cee
2.3.0p2.cee (default)

  2. …​ and then which versions are actually in use:

    root@linux# omd sites
SITE      VERSION        COMMENTS
mysite    2.3.0p2.cee    default version

    For these you will need to download the installation packages for the new operating system version. Important: The edition and version number of Checkmk must be exactly the same as the one used so far.

  3. Now uninstall all unused Checkmk versions. In the following command, the --purge parameter ensures that old configuration files are also erased.

    root@linux# apt-get remove --purge -y check-mk-enterprise-2.3.0p1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  check-mk-enterprise-2.3.0p1*
0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
After this operation, 884 MB disk space will be freed.
(Reading database ... 125980 files and directories currently installed.)
Removing check-mk-enterprise-2.3.0p1 (0.focal) ...
(Reading database ... 89444 files and directories currently installed.)
Purging configuration files for check-mk-enterprise-2.3.0p1 (0.focal) ...
Processing triggers for systemd (245.4-4ubuntu3.21) ...

3.2. Execution

And now for the hot phase, during which the Checkmk server is not available.

  1. Stop your Checkmk site:

    root@linux# omd stop mysite

  2. Rename the soft link pointing to the Checkmk installation you are using. The following command changes it from version to _version:

    root@linux# mv -v /opt/omd/sites/mysite/{,_}version

  3. Now you can uninstall Checkmk. In this case you keep existing configuration files:

    root@linux# apt-get remove -y check-mk-enterprise-2.3.0p2
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  check-mk-enterprise-2.3.0p2*
...

  4. Perform the upgrade of Linux according to the instructions provided by the respective distributor. In the case of Ubuntu, this would be:

    root@linux# do-release-upgrade

    If a restart is recommended, follow the recommendation before continuing to the next step.

  5. Deregister the soft link pointing to the Checkmk installation you are using. The following command changes it back from _version to version:

    root@linux# mv -v /opt/omd/sites/mysite/{_,}version

  6. Now install the package from Checkmk that matches the new version of the distribution you are using. In the case of Ubuntu, the following command is sufficient at this point:

    root@linux# apt install /tmp/check-mk-enterprise-2.3.0p2_0.jammy_amd64.deb

    When reinstalling Checkmk, follow the respective detailed installation instructions for your particular distribution.

  7. Restart your Checkmk site:

    root@linux# omd start mysite

4. Archive, reinstall and restore (upgrading out-of-place)

This method is often useful when using Checkmk in virtualized environments, in which as a preparation it is possible to 'pull up' a second server with the new distribution version and perform initial tests with this in a parallel operation. In addition, the flexibility of this approach is greater, because it is possible to simultaneously change the Linux distribution as well. Technically, this corresponds to the procedure in the event of a system failure.

4.1. Preparation

The most essential preparatory step is to download the appropriate Checkmk installation package for the new distribution version.

Get an overview of the Checkmk versions currently used by the sites:

root@linux# omd sites
SITE      VERSION        COMMENTS
mysite    2.3.0p2.cee    default version

Based on the information for the site(s) you will need to download the appropriate installation packages for the new operating system version.

Important: The Checkmk edition and version number must be exactly the same as the one used previously.

4.2. Execution

The steps, which are shown here in sequence, can often be performed partially in parallel, which helps to minimize downtime, for example, when using virtual machines or if a new hardware acquisition is scheduled anyway.

  1. Stop your Checkmk site — here this step is not mandatory, but is recommended for data consistency:

    root@linux# omd stop mysite

  2. Create an archive (in other contexts a backup) of the site. Our example uses a network share as the target, which will also be available later on the new installation.

    OMD[mysite]:~$ omd backup /mnt/someshare/mysite.tgz

  3. Now prepare the target system, and there configure the software required for Checkmk, for example, the email system. Use the same host name and IP address as with the source system.

  4. Install the previously downloaded Checkmk version, suitable for the new distribution.

    In the case of Ubuntu, the following command will suffice at this point:

    root@linux# apt install /tmp/check-mk-enterprise-2.3.0p2_0.jammy_amd64.deb

  5. Create a new site with the same name as the backed up site:

    root@linux# omd create mysite

  6. Now, as a site user, perform a restore from the archive:

    OMD[mysite]:~$ omd restore /mnt/someshare/mysite.tgz

  7. Restart your Checkmk site:

    OMD[mysite]:~$ omd start
On this page