Checkmk
to checkmk.com

1. Introduction

Should you ever be confronted with problems in Checkmk that you — with the help of this User guide — cannot solve by yourself, the official Checkmk Support and the Checkmk Forum are available which provide two excellent contact points. For both a precise description of the error or problem is naturally essential. In addition, certain specific information about your Checkmk environment is mandatory for a quick analysis and solution. The most essential information is of course the version and the edition of Checkmk that you are using. Depending on the situation, however, considerably more information may be required to be able to get to the bottom of your problem. In the past Checkmk support told you what information you needed to provide them with.

But under version 2.0.0 life can now be much easier as you have a tool that generates the required information — the Support diagnostics. Instead of the support providing you with a 'wish list' as mentioned above, and thus sending you on a scavenger hunt through the various components of Checkmk, with just a few clicks in the graphical user interface you can create a tailor-made package of information — referred to as the dump in this article. Here you decide for yourself whether you want to include configuration files that may possibly contain confidential information — or not, and if so to specify which ones.

2. Compile support information

2.1. Select site

After clicking on Setup > Maintenance > Support diagnostics you will see the following tidy screen:

support diagnostics

If you are using distributed monitoring, in the first field you can specify the Site from which you want to select the data to be collected.

2.2. General information

Under General information you will basically only find a notice that the information on the Checkmk version and the operating system in use will in any case be included. If you were to leave it at that — i.e. select none of the options — and start the collection of the data via Collect dump, you would get a file that contains only the following data:

general.json
{
    }, "arch": "x86_64",
    }, "core": "cmc",
    }, "edition": "cee",
    }, "os": "focal",
    "python_paths": [
        "/opt/omd/versions/2020.12.03.cee/bin",
        "/omd/sites/today/local/lib/python3",
        ...
        "/omd/sites/today/lib/python3"
    ],
    "python_version": "3.8.6 (default, Oct 22 2020, 16:40:44) \n[GCC 10.1.0]",
    "time": 1607002909.767516,
    "time_human_readable": "2020-12-03 14:41:49.767516",
    "version": "2.0.0b1"
}

2.3. Optional general information

Under Optional general information you will then find choices which you can compile in advance to suit your question, or which can be or which may be explicitly requested by the support personnel.

If you choose Local Files here, Checkmk will additionally create an overview of all files located in the ~/local/ directory of your site. This can be useful if your local customizations are incompatible with a recent update of Checkmk. All installed MKPs are also recorded here.

With the selection of OMD Config you can add the information on your configuration of OMD to the dump. These correspond exactly to what you would get on the command line with the omd config show command.

omd_config.json
{
    "CONFIG_ADMIN_MAIL": "",
    "CONFIG_APACHE_MODE": "own",
    "CONFIG_APACHE_TCP_ADDR": "127.0.0.1",
    "CONFIG_APACHE_TCP_PORT": "5000",
    "CONFIG_AUTOSTART": "on",
    "CONFIG_CORE": "cmc",
    "CONFIG_LIVEPROXYD": "on",
    "CONFIG_LIVESTATUS_TCP": "on",
    "CONFIG_LIVESTATUS_TCP_ONLY_FROM": "0.0.0.0 ::/0",
    "CONFIG_LIVESTATUS_TCP_PORT": "6557",
    "CONFIG_LIVESTATUS_TCP_TLS": "off",
    "CONFIG_MKEVENTD": "on",
    "CONFIG_MKEVENTD_SNMPTRAP": "on",
    "CONFIG_MKEVENTD_SYSLOG": "on",
    "CONFIG_MKEVENTD_SYSLOG_TCP": "on",
    "CONFIG_MULTISITE_AUTHORISATION": "off",
    "CONFIG_MULTISITE_COOKIE_AUTH": "off",
    "CONFIG_NAGIOS_THEME": "classicui",
    "CONFIG_NSCA": "off",
    "CONFIG_NSCA_TCP_PORT": "5667",
    "CONFIG_PNP4NAGIOS": "on",
    "CONFIG_TMPFS": "on"
}

If you check the Checkmk Overview checkbox, general information about all sites running on your Checkmk server will be included. This also creates a list of all installed Checkmk versions. And if the site selected above is the node for a cluster this fact will also be recorded here.

After activating the Checkmk Configuration files option you have the option of removing any confidential data from this part of the package — via the Pack only Low sensitivity files preference. All files you can select here come from the ~/etc/checkmk directory and its subdirectories. You can see a detailed list directly below the dropdown menu. With the Select individual files from list option you even have the option to include only specified files into the dump.

With this option you can also see which files have one of the confidentiality levels High (H), e.g. files with passwords, Medium (M), if they contain addresses or usernames for example, or finally Low (L).

support diagnostics sensitivity levels

Next, you have the option to add Performance Graphs of Checkmk Server to the dump. Especially in cases of problems with the performance of a Checkmk site these reports are almost always requested — so it is a good idea to include with such problems. The Support diagnostics will do the work for you to manually generate a number of reports as PDF files. Among others, the reports from the OMD mysite performance service for the last 25 hours and the last 35 days are generated.

2.4. Component-specific information

The Component specific information section again allows you to decide in a very granular way on which information from your global Checkmk settings, your hosts and folders, and your notification settings should be included in the dump.

Important: Depending on the configuration, the files you select here may contain confidential information, such as passwords. In normal operation, this data is protected by the fact that only the site user and administrators have access to it. If you make this data available to third parties for analysis purposes, you should proceed with great caution.

In the Global Settings sub-item you will find all global.mk-files of the individual components of your Checkmk site, such as the dynamic configuration daemon or the live status proxy daemon.

The information that can be selected via the sub-item Hosts and Folders can, among other things, help to find unfavorable rule sets and errors in the host configuration.

In the Notifications section you will find, in addition to the corresponding configuration files, an option for the selection of log files. In case of difficulties with your notifications, you — or ultimately the Checkmk support — can often find the source of the problem in these logs.

Note: In order to have as much detailed information as possible on the behavior of Checkmk in the logs, it may be necessary to change the log level in Checkmk for a short time. The corresponding settings can be found via Setup > General > Global settings. On this page, simply type logging into the Filter settings field and then set, for example, the log level for the Core to Debug. If you now simply let the site continue to run for a few minutes or if you repeat a reproducible error, the chances increase that information on this error show up in the log.

3. Support diagnostics via the command line

As is often the case in Checkmk, this task can be performed simply from a terminal. This can be done easily with the cmk command and the --create-diagnostics-dump option. For all options described above you can append the corresponding parameter to the command.

cmk --create-diagnostics-dump --local-files --omd-config --performance-graphs

The following options complete the output from the command:

--local-files

List of all installed, unpackaged, and optional files below ~/local. This also includes information about installed MKPs.

--omd-config

Contents of the ~/etc/omd/site.conf file.

--checkmk-overview

Information from HW/SW inventory node Software > Applications > Checkmk of the Checkmk server.

--checkmk-config-files FILE,FILE …​

All .mk and .conf files from the ~/etc/checkmk directory.

--checkmk-log-files FILE,FILE …​

All log files (.log and .state) from ~/var/log.

--performance-graphs

Performance graphs (e.g. CPU load, CPU utilization) of the Checkmk server.

The above, and all other options for the cmk command can as usual be found in the output from cmk --help.

4. Missing information in the dump

4.1. Current agent needed

To be able to output specific information in the Support diagnostics, you must make sure that the Checkmk servers have the agent with the version number of at least 2.0.0 installed. In particular, the information that comes from the HW/SW Inventory of the Checkmk server was not provided at all in older versions of the agent.

4.2. Label cmk/check_mk_server:yes

The Support diagnostics depend on the Checkmk servers in your environment being labelled accordingly. If you are missing some data in a dump, please check if your Checkmk servers are labelled with the cmk/check_mk_server:yes label.

On this page