Checkmk
to checkmk.com

1. Introduction

Updating Checkmk is a bit different than most other software packages that you may be familiar with. Why is that?

The reason is that Checkmk not only allows multiple independent sites to run on a single server, but it also allows multiple software versions to be installed simultaneously. With this system each site is assigned to an installed version of the software. To illustrate, we can take the following situation on a fictional server:

update1

Here the site mysite1 uses version 2.0.0p3.cee, but the sites mysite2 and mysite3 use version 2.0.0p1.cre. Although Checkmk-Version 2.0.0p1.cfe has been installed, it is currently not being used.

This example makes it clear that an update does not simply mean the installation of a new Checkmk RPM/DEB package on a server. An additional step is also required. Let us take the following situation as an example:

update2

Here the site mysite is to be updated to the Checkmk-version 2.0.0p3.cee. The first step is to download and install the appropriate RPM/DEB-Packet. This is performed exactly as with the initial installation. At first the newly-installed version will not be used by any site, and it will look like this:

update3

The second step will now be to update the site from 2.0.0p1.cre to the 2.0.0p3.cee. This is performed with the omd update command, which we will discuss in detail below:

update4

Following the update, the (likely) no longer required version 2.0.0p1.cre can be deleted by uninstalling the appropriate package.

2. Before the update

If you are planning to update Checkmk from version 2.2.0 to version 2.3.0, you should first read the Update to version 2.3.0 article, in which we have compiled the most important issues you should consider before and after such an update.

However even if you have already installed a 2.3.0 version and want to update to a new stable patch version of 2.3.0, the topics described in the following sections may still be relevant.

2.1. Update to major versions

When updating to a higher major version, you must always update step-by-step through any available intermediate versions until the target version has been reached, and not just skip over any intermediate versions. For example, if you want to update from version 2.1.0 to version 2.3.0, first update to the intermediate version 2.2.0. The reason for this procedure is simple: there are sometimes simply so many changes between two major versions that skipping versions can cause problems.

The omd update command also allows an 'update' to a lower version. This procedure is only intended for regressions. After such an update in the reverse direction, many adjustments will be required to make the configuration and runtime environment compatible again — especially, but not only, in the case of an 'update' to a lower major version. We therefore strongly advise against such a procedure — and will also no longer provide support in the case of an update to a lower version.

2.2. Incompatible and obsolete MKPs

Your monitoring system can be extended quite easily and conveniently using the Checkmk extension packages (MKPs). On the one hand, it is possible that some older MKPs are no longer being maintained and thus may no longer be compatible with newer versions of Checkmk. On the other hand, we keep adding new plug-ins and functional extensions to Checkmk, which is why MKPs sometimes become obsolete. The functionality of those plug-ins and extensions is simply provided by Checkmk itself as standard.

For this reason, when preparing for an update it is strongly recommended that you review all installed MKPs. This will prevent incompatible packages from interfering with the update, or resulting in duplicate or at least very similar services following the update.

To do this, check your installed MKPs against our Catalog of Check Plug-ins and remove any packages containing functions that are now provided natively by Checkmk. You can also use this opportunity to remove MKPs that may have only been installed for a test run. A list can be found in the Setup menu under Maintenance > Extension packages. On the command line, you can display the installed extensions with the mkp list command. Check the output of this command for extensions that are no longer needed or that you cannot even identify.

Checkmk supports the installation of MKPs for a version newer than the currently running version in preparation for future updates. When performing an update, the package for the lower Checkmk version is then disabled and the package for the higher version is enabled. Details are explained in the article on using MKPs.

Caution: If you have made local changes to files that originally came from MKPs, repack the MKP after increasing the version number. During an update, files that have been otherwise modified will be overwritten by those contained in the MKP.

2.3. Local files

Local files allow you to customize and extend the functionality provided by Checkmk. These files are located in the local part of the site directory structure, i.e. in ~/local. Local files can cause problems when updating, as they may no longer match the new Checkmk version.

Since it is not possible for Checkmk to intercept and handle local customizations and any third-party extension during an update, you should check your Checkmk site before an update to see if and which local files you are using.

Get an overview of the local files of your Checkmk site by executing the following command as the site user (where the -L option ensures that symbolic links are also followed):

OMD[mysite]:~$ find -L ~/local -type f

In a fresh installation of Checkmk, you will currently only see a file called README.TXT listed. Anything beyond that should be at the top of your troubleshooting list in case you have problems updating.

Ideally, local files are already fully packaged in MKPs. Use mkp find to identify unpackaged files. For further details on creating packages, see our article on Checkmk extension packages. Once packaged, each extension can be deactivated or reactivated as a complete element.

2.4. Backup and test run

We don’t need to remind you of the importance of creating a backup immediately before any update, so that you don’t risk losing too much of your monitoring history in the event of a failure during the update. What is relevant at this point is that a regular backup can also serve you well for test runs of a pending update. This practice enables you to restore the backup under an alternative name — and then use the newsite site to test the update before it goes live:

root@linux# omd restore newsite /path/to/backup

Alternatively you can also copy your site via omd cp. For this, however, a live site must be stopped for a short time:

root@linux# omd stop mysite && omd cp mysite newsite && omd start mysite

Then run the update on this new, cloned site first, for example, to check that the local changes mentioned above have been made in the new environment. If the tests with the cloned site were successful, you will usually want to delete or at least stop it before the actual update of the production site for space and performance reasons.

Tip

Since Checkmk 2.3.0, if an update fails, the configuration status prior to the update will be restored. An update can fail due to an unexpected internal error, but also due to user input during the update process, for example by selecting the menu option abort or by pressing the key combination CTRL+C. Restoring the configuration does not replace a complete backup, but in many cases helps to reduce downtimes.

3. Updating Checkmk

3.1. Installing new versions

As described in the introduction, the first step with an update is the installation of the desired new version of Checkmk. This is achieved in exactly the same way as with the initial installation — it will however proceed somewhat more quickly since most of the dependent packages have already been installed. In the following example we are installing the package for Ubuntu 22.04 (Jammy Jellyfish):

root@linux# apt install /tmp/check-mk-enterprise-2.3.0p1_0.jammy_amd64.deb

Note: When installing a local package via apt install, you have to state the pull path to the .deb file.

A list of your installed Checkmk versions can be displayed at any time with the omd versions command:

root@linux# omd versions
2.1.0p42.cre
2.2.0p22.cee
2.3.0p1.cee (default)

One of these listed versions is marked with (default). This default version will be used automatically when creating new sites, as long as no other version is specified with omd -V myversion create mysite. The current default version can be queried with omd version, and it can be altered with omd setversion:

root@linux# omd version
OMD - Open Monitoring Distribution Version 2.3.0p1.cee
root@linux# omd setversion 2.2.0p22.cee
root@linux# omd version
OMD - Open Monitoring Distribution Version 2.2.0p22.cre

The default version plays no role when managing existing sites. The omd command always starts with the version appropriate to the site for which the command is called.

A listing of the current sites and the versions they use is provided by the omd sites command:

root@linux# omd sites
SITE             VERSION          COMMENTS
mysite           2.2.0p22.cee
test             2.3.0p1.cee      default version

3.2. Performing the update

Once the desired new version has been installed, the site can be updated. No root-permissions are required for this. The best way to do this is as a site user:

root@linux# omd su mysite

Ensure that the site has been stopped:

OMD[mysite]:~$ omd stop

The update — in effect switching to a different version — can now simply be performed with the omd update command:

OMD[mysite]:~$ omd update

If more than one target version is available, a selection list will open:

update omd update 2

In the following dialog box, confirm the selected update to the new version:

update omd update 3

To be on the safe side, when you perform an edition upgrade from an Checkmk Raw to one of the commercial editions at the same time as the version update, you will be reminded of this fact again:

update raw to enterprise

An important part of an update is the refreshing of the originally provided configuration files. Here changes that had possibly been made to these files by the user will not simply be discarded, instead they will be merged. This functions very much like version control systems which attempt to amalgamate changes made to a single file simultaneously by multiple developers.

Occasionally — when the changes affect the same location in the file — that won’t function, and a conflict occurs. How you can solve such conflicts will be explained later below.

The update provides a listing of all modified files and directories (shortened in the following example):

2024-04-09 15:55:04 - Updating site 'mysite' from version 2.2.0p23.cee to 2.3.0p1.cee...

 * Installed dir  etc/default
 * Updated        etc/mk-livestatus/nagios.cfg

...
 * Vanished       etc/jmx4perl/config/websphere
 * Vanished       etc/jmx4perl/config
Creating temporary filesystem /omd/sites/mysite/tmp...OK
ATTENTION
  Some steps may take a long time depending on your installation.
  Please be patient.

Cleanup precompiled host and folder files
Verifying Checkmk configuration...
 01/05 Rulesets...
 02/05 UI extensions...
 03/05 Agent based plugins...
 04/05 Autochecks...
 05/05 Deprecated .mk configuration of plugins...
Done (success)

Completed verifying site configuration. Your site now has version 2.3.0p1.cee.
Executing update-pre-hooks script "01_init_state_creation.py"...OK
Executing update-pre-hooks script "01_mkp-disable-outdated"...OK
Executing update-pre-hooks script "02_cmk-update-config"...
-| ATTENTION
-|   Some steps may take a long time depending on your installation.
-|   Please be patient.
-|
-| Cleanup precompiled host and folder files
-| Verifying Checkmk configuration...
-|  01/05 Rulesets...
-|  02/05 UI extensions...
-|  03/05 Agent based plugins...
-|  04/05 Autochecks...
-|  05/05 Deprecated .mk configuration of plugins...
-| Done (success)
-|
-| Updating Checkmk configuration...
-|  01/25 Cleanup microcore config...
...
-|  25/25 Update core config...
-| Generating configuration for core (type cmc)...
-| Starting full compilation for all hosts Creating global helper config...OK
-|  Creating cmc protobuf configuration...OK
-| Done (success)
OK
Finished update.

If the Completed verifying site configuration line highlighted in the above output is displayed, the site has been switched to the new version:

OMD[mysite]:~$ omd version
OMD - Open Monitoring Distribution Version 2.3.0p1.cee

… and can then be started:

OMD[mysite]:~$ omd start

3.3. Incompatible changes

Software development of course consists of changes. Because we are always actively working to keep Checkmk modern, sometimes cutting dead weight and making changes that turn out to be incompatible is unavoidable. That means that when updating it may possibly be necessary to adapt your configuration, or you should at least check it.

A typical example of such a situation is with new check plug-ins which replace existing plug-ins. If you use one of the affected plug-ins, a fresh service discovery will be required on the affected host.

An overview of all changes in Checkmk, including a search function, can be found online in our Werks.

Even more practical however is the search function built into Checkmk. After you have logged in to the site, you will find a link highlighted in red at the top of the Help menu with the number of incompatible and not yet confirmed changes:

“Help menu with link to the list of incompatible changes.”

Checkmk keeps track of incompatible changes that occurred during the update to the current version and asks you to review and then acknowledge these:

“List of incompatible and open changes.”

If you open this page via the red link in the Help menu, you will only see the 'Werk' (i.e. the changes) for which something needs to be done and which are therefore marked with Incompatible - TODO. You can call up each Werk individually, view it, confirm it with a mouse click — and thus successively reduce the number of open incompatible changes. In addition, the Help > Change log (works) menu item gives you access to the complete history of changes in the current major version.

3.4. The update in detail

Are you curious about what exactly is happening ‘under the hood’ of an update? Or have data conflicts appeared when omd update is running? If so, here is some further reading.

Three actions take place during omd update:

  1. The refreshing of the default files under ~/etc/ and ~/var/ – i.e., files created by omd create.

  2. The switching of the active version to the target version by changing the symbolic link version which is found in the site directory.

  3. Post-processing by various packages (e.g., Checkmk). In particular, an activate changes will be automatically executed in order to generate a valid configuration for the core.

Updating files, merging changes

The first step is by far the most comprehensive. Here Checkmk demonstrates a big advantage in comparison to the typical software installation — Checkmk helps you to adapt all of the standard configuration files to the prerequisites of the new version. This resembles the procedure for updating a Linux distribution, but goes further in the implementation. Checkmk can handle a multiplicity of situations, for example:

  • The merging of file changes made by the update with any changes made locally by the user.

  • Files, directories and symbolic links which are obsolete in the new version, or which have been deleted by the user.

  • Changes to permissions.

  • Changes to a file type (a symbolic link derived from a file or directory, or vice versa).

  • Changes to the target of a symbolic link.

Checkmk always ensures that your local changes are retained, and that all of the changes required by the new version are simultaneously implemented.

Merging and conflicts

If the new version requires changing a configuration file on which the user has also made changes, Checkmk automatically attempts to merge both sets of changes. This is achieved using the same methods as used by version-control systems.

The fewest problems are experienced when your and Checkmk’s changes have a clear physical separation in the text (at least a few lines apart). The merge will then be effected automatically, and without needing the user’s intervention.

If two changes ‘collide’ because they both affect the same location in the data, Checkmk cannot and will not decide which of the changes is more important. In such a situation you will be notified as a user and can interactively resolve the conflict:

omd update

In the situation shown above, you now have the following options:

d

This shows the differences between the new default version and your version of the file in the form of a ‘unified diff’ (diff -u).

y

This is similar to the above, but based on the preceding default version shows which changes you have made to the file.

n

This third option in effect ‘closes the triangle’ by showing the changes which Checkmk intends making to the file.

e

Resolve the conflict manually in an editor.

t

By selecting t, your original file – without the already successfully-merged changes – will be opened in an editor. Now edit the file in order to bypass possible conflicts. Once the editor has been closed Checkmk will reattempt the merge.

k

Here you can decide whether to accept the data ‘as is’. The successfully inserted changes are retained. Apart from this the file remains as customized by the user.

r

With this you can fall back to the old version of your file, and go without Checkmk’s update for this file. Any customizations that may be required must be performed manually.

i

Install the new default file version: your changes in the old file will be lost.

s

If you are uncertain, you can open a shell with s. You will find yourself in a directory containing the relevant file, and there can get a picture of the situation. Quit the shell with CTRL+D in order to proceed with the update.

a

Abort the update. The site retains the old version and its previous configuration is restored. You can start a new update attempt at any time.

Further conflict situations

Alongside the content-merging of files there is a whole series of further situations in which Checkmk requires your decisions. Some of these are very unusual situations, that nevertheless need to be handled correctly. In these cases Checkmk will always give you the choice of keeping your version, or of adopting the new default version. What is more, there is always the option of aborting an update, or of opening a shell. Examples of such ‘difficult’ situations are:

  • conflicting changes to file types (e.g., when a file is replaced by a symbolic link)

  • conflicting changes to file permissions

  • changed files that are not required by the new software version

  • files, directories or links created by a user, which conflict with a new version’s files/directories/links

Explanation of the output during an update

The update procedure will always output a line of explanation when it makes a change to a file automatically. The following situations are possible – files are referred to here, but this also applies analogously to links and directories:

Updated

A file has been changed with the new version. Since you have not made a change to the file, Checkmk simply installs the new default version of the file.

Merged

A file has been changed with the new version, and at the same time the user has made other changes to the file. Both versions of the file can be merged into a single version without conflict.

Identical

A file has been changed in the new version, and at the same time the user has already made identical changes to the file. Checkmk must not perform any action.

Installed

The new version includes a new configuration file which has just now been installed.

Identical new

The new version includes a file, an identical copy of which the user has already installed.

Obsolete

The new version has obsoleted a file (also applies to a link or a directory). The user has anyway already deleted it. No action.

Vanished

Another file is obsolete in the new Checkmk, and the user has neither deleted nor changed the existing version. Checkmk deletes this file automatically.

Unwanted

The user has deleted a file which is normally present. Because the version in the new Checkmk has no changes from the last version of the file, Checkmk allows the file to be absent.

Missing

The user has already deleted a file, but in the new Checkmk this file contains changes from the previous version. Checkmk installs the new file, and logs a notification of this action to the user.

Permissions

Checkmk has updated a file’s permissions because different permissions are set in the new version.

3.5. Updating without user interaction

Would you like to automate Checkmk’s software updates? You may at first have difficulties with the interactive responses from omd update. There is a simple solution for this scenario: the command has options that have been especially conceived for use in scripts:

  • The options -f or --force directly following omd inhibit all types of "Are you sure…​ ?" questions.

  • The option --conflict= directly following update determines the desired behavior if a file conflict occurs.

Possible values for --conflict= are:

--conflict=keepold

In the event of a conflict, the user’s own modified version of the file is retained. It is however possible that Checkmk may not be executable, and that manual rectification will be required.

--conflict=install

In the event of a conflict, the new standard version of the file will be installed. With this, local changes to the file will be at least partly lost.

--conflict=abort

In the event of a conflict the update is canceled and the previous configuration of the old version is restored,

--conflict=ask

This is the standard procedure, so in this form the option is actually superfluous.

Below is an example of the complete command for an automated update to version 2.3.0p1.cee of mysite:

root@linux# omd stop mysite ; omd -f -V 2.3.0p1.cee update --conflict=install mysite && omd start mysite

Through the && before omd start a restarting of the site will be prevented if the omd update is aborted by an error. Replace the && with a semicolon (;) if a start should definitely be attempted even in such a situation.

If you are certain that only a single Checkmk site is running on the server, the name to be used in a shell script can simply be trapped in a variable:

root@linux# omd sites --bare
mysite
root@linux# SITENAME=$(omd sites --bare)
root@linux# echo $SITENAME
mysite

This enables the above line to be independent of the site’s name. For example, a small shell script could look like this:

update.sh
#!/bin/bash
SITE=$(omd sites --bare)
VERSION=2.3.0p1.cee

omd stop $SITE
omd -f -V $VERSION update --conflict=install $SITE  && omd start $SITE

4. Updating in distributed environments

There are two different procedures for performing the update of all sites included in a distributed monitoring — i.e. the central site and any remote sites controlled by it.

Important: Whichever approach you choose, as the very first step you should create backups of all sites in the environment.

The preferred, safest procedure is to update in one go, in which you perform the following steps:

  1. First, stop all sites

  2. Then perform the update for all sites

  3. Restart the updated sites

If this is not possible — for example, because the environment is distributed across sites time zones and with different supporting teams — a temporary mixed operation can be implemented under strict conditions. The versions may differ by no more than one version for major updates, and always assumes a specific patch level for the current (existing) version.

It is essential to follow exactly this sequence: First, update all remote sites, and only then update the central site. This ensures that at no time does a configuration created by a newer Checkmk version end up in an older Checkmk version.

The following table shows the possible combinations when updating from 2.2.0 to 2.3.0:

Central site Remote site Permitted? Notes

2.2.0

2.2.0

Yes

State before updating all sites.

2.2.0

2.3.0

Yes

Minor functional losses are to be expected during the update, so only operate as a mixed environment for a short period of time. There is no danger for data and configuration.

2.3.0

2.2.0

No

Caution: With a centralized configuration there is a risk of irreparably damaging remote sites in this situation. Avoid this condition at all costs!

2.3.0

2.3.0

Yes

State after updating all sites.

4.1. Technical background

The technical reason for the updating approach as described above lies in the protocols used: The central site accesses the data on the remote sites primarily by reading via Livestatus, and in the case of a central configuration, with additional write access via a non-public HTTP API. In both situations, it is the case that new versions introduce real supersets of the protocols used. Thus, an older central site uses only a true subset of the functionality of the newer remote sites. If the central site were updated first, it might issue API calls or Livestatus requests to the remote sites that these do not yet 'understand'.

The maximum version difference of one major version again results from the fact that the removal of interfaces is accompanied by a grace period of exactly one version. For example, the Web API was already no longer used internally by Checkmk 2.1.0, but its removal did not occur until version 2.2.0. For this reason, a 2.1.0 central site works with 2.2.0 remote sites, but a 2.0.0 central site will not work with 2.2.0 remote sites.

4.2. Extension packages for use in centralized configurations

To facilitate such phased updates, Checkmk provides the ability to store extension packages with the same name in different versions — one matching the older central site, one matching the newer remote sites — for example. The appropriate version will be activated for each site. Details are described in the article on extension packages (MKPs).

4.3. Cascading Livestatus

With the viewer sites extension, the viewer sites may be updated only after the sites whose data they display. If a viewer site only shows data from remote sites, it may be updated as soon as these are updated. If, on the other hand, it also shows data from the central site, the viewer’s update may only take place last.

5. Updating a Docker container

The update of a Checkmk site in the Docker container is very simple. The following are the only requirements:

  • The container is not deleted when the container is stopped — i.e., the --rm option was not used at startup.

  • You know the ID of the volume for the container. Normally you should have given its storage a unique ID when you started the container. If you are unsure of your volume’s ID, you can retrieve information about the container named myContainer with the command docker inspect myContainer.

If you followed the installation guide for Checkmk in Docker you should automatically meet the requirements.

The update process is performed in 3 steps:

  1. Stop the container. If the Checkmk container is called myContainer, the command will be: docker stop myContainer.

  2. Remove the container. The command is: docker rm myContainer.

  3. Start a new container with the command docker container run with the desired version, and mount the known volume. If your volume is called myVolume, the corresponding option is -v myVolume:/omd/sites. All options of the command can be found in installation guide for Checkmk in Docker.

Checkmk will then automatically do the rest — updating and starting your Checkmk site. Afterwards you will be able to log in as usual.

6. Upgrades

Upgrades from CRE Checkmk Raw to one of the commercial editions or from one of the commercial editions to another with greater functionality are straightforward at any time. The procedure is essentially always the same: Install the desired package and switch the relevant sites with omd update. If you upgrade to one of the commercial editions, you must license it after the upgrade.

When upgrading a commercial version to one with a larger range of functions, the license must always be re-entered after the upgrade. This also applies to cases in which your sales contact has assured you that you can already use your 'lower' Checkmk environment with the 'higher' license following a license upgrade: When upgrading, the license state is reset (the history is kept), thus the license must be entered again.

6.1. Upgrading Checkmk Raw to one of the commercial editions

This chapter primarily covers an upgrade to Checkmk Enterprise. You can also upgrade to Checkmk Cloud in one step, but in this case, also refer to the notes in the following section.

Since the commercial editions have quite a few additional modules and features, there are a few things to keep in mind following any upgrade. The crucial point is that when creating new sites in Checkmk Raw or one of the commercial editions, different default settings are set.

Nagios vs. the CMC

Since Checkmk Raw only supports Nagios as its core, this is the default setting for sites created with Checkmk Raw. This setup will be preserved when upgrading to CSE Checkmk Enterprise. This means that after an upgrade you will initially continue to run with Nagios as the core. A migration to the CMC is done with omd config and is described in its own article Migration to the CMC.

The RRD format

The commercial editions support an alternative format for storing historical measurement data, which generates significantly less disk I/O. This format is automatically preset for new commercial edition sites. Again, Checkmk Raw sites are not automatically converted during an upgrade. How to switch the data formats is described in a separate section in the article on measured values and graphing.

Further differences

To take full advantage of Checkmk Enterprise, refer to the overview of differences between Checkmk Raw and Checkmk Enterprise.

6.2. Upgrading from Checkmk Enterprise to Checkmk Cloud

When it comes to the monitoring core and the notification system, there are no differences between Checkmk Enterprise and Checkmk Cloud. Depending on the focus of the deployment, you will often use the larger feature set only when adding new hosts. In some places, however, it is still advisable to review the existing settings.

For a complete overview of the additional functionality, see the article on Checkmk Cloud.

Check plug-ins for cloud services

When you monitor Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP), the services in existing hosts reserved for Checkmk Cloud will initially not be enabled. You can enable these services in the XYZ services to monitor rule (where XYZ is the name of the cloud platform). Then perform a service discovery on these hosts to find the services that will now be available.

Agent Controller in push mode

With the ability to directly monitor hosts that can reach the Checkmk server but are not accessible from it, the need for homegrown solutions with data source programs is eliminated in many cases. You can switch these hosts to push mode to enable a direct monitoring.

6.3. Upgrading editions in distributed environments

Note that in distributed environments, the version update must always be performed first before the edition upgrade can follow. A different sequence or a crossgrade (update of the version and upgrade of the edition in a single action) is not supported. As a general procedure, we recommend the offline upgrade, where you can jump from any lower edition to any higher edition.

For two frequently requested upgrade scenarios (Checkmk Raw to Checkmk Enterprise and Checkmk Enterprise to Checkmk Cloud), mixed operation is possible for a limited period of time. While the offline upgrade works with all versions, the two online scenarios require at least Checkmk 2.2.0p23.

Offline upgrade (all combinations)

As mixing editions is only possible in a few combinations, we generally recommend upgrading the edition offline. To do this, proceed as follows:

  1. Stop all sites.

  2. Perform the upgrade of the central site.

  3. If it is desired (and an abundance of notifications is not a problem), the central site can be restarted immediately.

  4. Now it’s time to upgrade the remote sites. You can do this in parallel and restart the remote sites immediately following their upgrading.

Of course, you can wait until all sites have been upgraded before restarting them, which will somewhat reduce the number of notifications generated.

Checkmk Raw to Checkmk Enterprise (online)

Checkmk only allows mixed operation of Checkmk Enterprise as a central site with Checkmk Raw or Checkmk Cloud as remote sites. For the upgrade to Checkmk Enterprise, this means that the central site receives the update first. During mixed operation, configuration should not be transferred from the central site to the remote sites that have not yet received an upgrade. We therefore recommend preventing the configuration from being changed for the duration of mixed operation on the central site via Setup > General > Read only mode. Theoretically, this mixed operation of Checkmk Enterprise as the central site with Checkmk Raw as the remote site can last as long as required.

This results in the following sequence for the upgrade:

  1. Upgrade the central site to Checkmk Enterprise.

  2. Enter and submit the subscription data as described in the licensing article. The services provided by all remote sites are treated equally when calculating the subscription, i.e. during mixed operation, services of Checkmk Raw remote sites are already billed as Checkmk Enterprise.

  3. Gradually update the remote sites.

  4. If you have not deactivated the transfer of the configuration globally, but separately for each remote site, you can reactivate the transfer of the configuration for each remote site that has received the upgrade.

  5. In distributed monitoring without distributed setup, the remote sites must also be licensed immediately after the upgrade.

After upgrading all sites, you can activate Checkmk Enterprise specific features.

Tip

Why do we recommend not to synchronize the configuration during the upgrade?

Remote sites actually discard settings that they 'can’t do anything with'. This does not break anything, but it can mess things up. Let’s say you activate a Checkmk Enterprise specific setting — for example regular scheduled downtimes — before all remote sites have received the upgrade. In this case, Checkmk Raw sites will discard the setting, which means that it will not be available even after the upgrade. The setting will only be available once it has been changed again after the upgrade.

If it is unavoidable not to synchronize the configuration during the upgrade, check the consistency of Checkmk Enterprise specific settings after the complete upgrade.

Checkmk Enterprise to Checkmk Cloud (online)

As already mentioned, Checkmk only allows mixed operation of Checkmk Enterprise as a central site with Checkmk Raw or Checkmk Cloud as remote sites. For the upgrade to Checkmk Cloud, this means that the central site receives the update last. The upgrade of the entire environment must be completed within 30 days. During mixed operation, configuration can be transferred from the central site to the remote sites.

This results in the following sequence for the upgrade:

  1. Upgrade the remote sites to Checkmk Cloud. Important: A mixed operation with Checkmk Enterprise as the central site is only possible in the 30-day 'Trial' license state. Do not store any Checkmk Cloud subscription data yet!

  2. Store the subscription data for Checkmk Cloud in the central site under Checkmk Enterprise.

  3. Upgrade the central site to Checkmk Cloud.

  4. In distributed monitoring without distributed setup, the remote sites must now also be licensed immediately after the upgrade.

If you want to upgrade directly from Checkmk Raw to Checkmk Cloud, you must upgrade the central site to Checkmk Enterprise as an intermediate step: First upgrade the central site from Checkmk Raw to Checkmk Enterprise, followed by the upgrade of the remote sites directly from Checkmk Raw to Checkmk Cloud. Finally, upgrade the central site from Checkmk Enterprise to Checkmk Cloud.

7. Downgrades

Downgrades between editions are also possible. A downgrade is a more complex and thus time-consuming action, since some features may not work in the target edition, and will need to be manually deactivated and replaced by a possibly less efficient or less convenient alternative.

The actual downgrade must be carried out with the command omd update - just like an update or upgrade. The details can be found in the section Performing the update.

If you try to downgrade from Checkmk Cloud to Checkmk Enterprise, for example, Checkmk will verify whether this is actually your intention:

update downgrade ce se

As soon as you confirm this dialog with yes, the downgrade starts immediately. Please find out below what you need to do before this downgrade.

Downgrades other than those described below are not intended and are not supported by us, for example a downgrade of Checkmk MSP. Instead, we recommend starting with a fresh site in such a case.

7.1. Downgrading from Checkmk Cloud to Checkmk Enterprise

In preparation for the downgrade from Checkmk Cloud to Checkmk Enterprise, you must make at least the following changes:

  • Set hosts that operate in push mode to pull mode. Otherwise Checkmk will not receive monitoring data from them and their associated hosts will usually become stale.

  • Reconfigure the agent packages for folders to stop auto-registration. Then rebake the agent packages.

Furthermore, some cloud services and dashboards will no longer be available. As a result, you will need to clean up vanished services.

If under Checkmk Cloud you used the Grafana plug-in from the 'Grafana Store', you will need to replace it with one installed from the zip archive.

An overview of the differences between Checkmk Cloud and Checkmk Enterprise is provided by the Checkmk Cloud article.

7.2. Downgrading from Checkmk Enterprise to Checkmk Raw

In preparation for downgrading from Checkmk Enterprise to Checkmk Raw, you will need to make at least the following changes:

  • Change the RRD database format with the Configuration of RRD databases of hosts rule to Multiple RRDs per host/service. Besides slight performance disadvantages, it should be noted here that conversion of existing data is not included, so historical monitoring data will no longer be visible.

  • Switch the monitoring core from CMC to Nagios — in the first place this change is likely to result in performance disadvantages.

In addition, some dashboards, graph settings, notification plug-ins and special agents are no longer available. With the Checkmk Enterprise article you can determine how much Checkmk Enterprise functionality will be lost in the event of a downgrade to Checkmk Raw and where you may need to make further adjustments.

7.3. Edition downgrades in distributed environments

The downgrade scenarios presented in the two previous sections can also be carried out in distributed environments. Note that in distributed environments, the version update must always be performed first before the edition downgrade can follow. A different sequence or a crossgrade (update of the version and downgrade of the edition in a single action) is not supported.

There is no downgrade scenario in which Checkmk supports a mixed operation between differing editions. For this reason, we strongly recommend that you perform the edition downgrade offline. To do this, proceed as follows:

  1. Stop all sites.

  2. Perform the downgrade of the central site.

  3. If it is desired (and an abundance of notifications is not a problem), the central site can be restarted immediately.

  4. Now it’s time to downgrade the remote sites. You can do this in parallel and restart the remote sites immediately following their downgrading.

Of course, you can wait until all sites have been downgraded before restarting them, which will somewhat reduce the number of notifications generated.

8. Uninstalling Checkmk

8.1. Overview

The uninstallation of no longer required Checkmk versions is performed using the operating system’s package manager. To do this, enter the installed package’s name – not the file name of the original RPM/DEB file.
Important: Only delete Checkmk versions that are no longer being used by any site!

Checkmk sites that are no longer required can simply be removed with omd rm (thereby deleting all data as well!):

root@linux# omd rm mysite

8.2. SLES, RedHat, CentOS

Here is how to identify which Checkmk packages are being used in RPM-based systems:

root@linux# rpm -qa | grep check-mk
check-mk-enterprise-2.3.0p1-el9-38.x86_64.rpm
check-mk-raw-2.2.0p25-el9-38.x86_64.rpm
check-mk-raw-2.1.0p34-el8-38.x86_64.rpm

The deletion is performed with rpm -e:

root@linux# rpm -e check-mk-raw-2.1.0p34-el8-38.x86_64.rpm

8.3. Debian, Ubuntu

Use the below to identify which packets are installed:

root@linux# dpkg -l | grep check-mk
ii  check-mk-enterprise-2.3.0p1  0.jammy  amd64  Checkmk - Best-in-class infrastructure & application monitoring
ii  check-mk-raw-2.2.0p25        0.jammy  amd64  Checkmk - Best-in-class infrastructure & application monitoring
ii  check-mk-raw-2.1.0p34        0.jammy  amd64  Checkmk - Best-in-class infrastructure & application monitoring

The uninstallation is performed with dpkg --purge:

root@linux# dpkg --purge check-mk-raw-2.1.0p34
(Reading database ... 567850 files and directories currently installed.)
Removing check-mk-raw-2.1.0p34 (0.jammy) ...
...

9. Fault Diagnosis

If an error occurs when updating Checkmk, it is usually due to one of the following three causes, which have already been mentioned in the previous chapters:

10. Files and directories

The files and directories relevant to this article can be found here. As always, paths that do not begin with / apply after the site’s home directory (/omd/sites/mysite).

File path Function

~/version

Symbolic link to the installation of the Checkmk version used by this site.

/omd/versions

In this directory a subdirectory exists for every installed Checkmk version. The files belonging to root and are never changed.

/omd/sites

In this directory, for every site there is a home directory containing its configuration files and variable data. This data belongs to the site’s user, and can be changed by configuration and operations.

/usr/bin/omd

Management command for Checkmk sites. This is a symbolic link to the default version’s bin-directory. When a particular site is accessed the omd-command substitutes itself with that of the appropriate version.

On this page