Checkmk
to checkmk.com

1. Introduction

The most important task when setting up a monitoring tool is to manage the systems to be monitored — the hosts. This is not just a matter of ensuring that they are entered with the correct master data (e.g. host name, IP address). Settings for the monitoring (e.g. notifications, thresholds, etc.) also require regular maintenance.

Checkmk was designed from the beginning for environments with very many hosts. To make the configuration of such environments manageable for the user, Checkmk follows a different approach from all other systems that have been developed out of the Nagios ecosystem. The main Checkmk principles are:

In general, it has proven useful to think about an ordering system first and then fill it with content. To learn about the possibilities Checkmk offers to bring order to your hosts, see the article on structuring hosts.

2. Folders and inheritance

Everyone who works with computers knows the principle of files and folders. Checkmk uses an analogous principle for the administration of the hosts, which effectively take over the role of the files. Since folders can themselves be contained in folders, the result appears as a tree structure.

2.1. Folder hierarchy

The user is completely free in the design of their own tree structure. Any form of differentiation is possible. However, there are three common criteria for the structure of the host tree:

  • Location (e.g. Munich versus Shanghai).

  • Host type (e.g. switch versus load balancer)

  • Organization structure (e.g. database group versus networker).

The criteria can of course be mixed, split by location in the first level of the tree and then by host type in the second for example.

If you like simplicity, you should put hosts only in the 'leaves' of the tree (even if Checkmk allows hosts in middle folders). The following example shows a simple tree structure by host type: Hosts A, B, and C are placed in the Servers folder, and D, E, and F are placed in the Network folder:

Illustration of a folder structure with two subfolders.

2.2. Inheriting attributes

If you build the tree cleverly, you can use it to inherit attributes in a meaningful way. This is especially useful for those attributes that are common to large groups of hosts, such as the SNMP community or host tags that you use to determine whether the host should be monitored by a Checkmk agent or by SNMP.

The following example shows the inheritance of the tag groups 'Criticality' (with the values prod and test) and 'Checkmk agent / API integrations' (with the values tcp and no-agent). The tcp auxiliary tag is automatically set when the Checkmk agent and/or an API integration is selected, while no-agent is the option of choice when monitoring via SNMP.

Illustration of a folder structure with attributes assigned at different levels.

Attributes defined further down the tree always take precedence. So values defined directly at the host override anything coming from the folders. In the above example, this results in prod and tcp for host A, prod and no-agent for D, and test and no-agent for host F because of the explicitly assigned attribute.

A major advantage of this scheme over the widely used 'copy & paste' approach of database-oriented configuration systems is that any specified attributes will be received by hosts added in the future. This makes work easier for you and your colleagues: Just drop a new host into the appropriate folder and all the predefined attributes will automatically be correct.

2.3. Creating folders

How to create folders is explained in the Beginner’s Guide.

3. Creating and editing hosts in the Setup menu

To manage folders and hosts, go to Setup > Hosts > Hosts:

Folder view 'Main' with multiple subfolders.

Here in the default view you can see an (initially empty) overview of the folders and below these, in tabular form, the hosts already present in the current folder. Creating a new host with the icon new Add host action button, the Icon to clone a list entry. clone or the Icon to edit a list entry. edit of an existing host will bring you to the Properties of host page. This lists the attributes of the host. These attributes are grouped into several sections, the most important of which we present here.

3.1. The host name

The Hostname field is used everywhere within Checkmk to uniquely identify the host. Whenever possible, you should use the host’s DNS name. If the DNS name is too bulky, an easier to recognize alias name can be assigned later.

Dialog with host properties: the host name.

Changes to the host name are in principle also later possible, however since the host name is used in many places in Checkmk, later changes are more complex and time-consuming when monitoring data has already been accumulated by the time the change is to made.

3.2. Alias and IP address

You can assign an alternative, descriptive name for the host under Alias, which is displayed in many places in the GUI and in reports. If you do not assign an alias, the host name will be used instead.

Dialog with properties of a host: alias and IP address.

You do not necessarily have to specify an IP address. Four options are available for configuring the IP address, which also determine when the name is resolved:

OptionProcedureWhen to perform name resolution?

1.

You do not specify an IP address. The host name must be resolvable by DNS.

At activating the changes.

2.

You enter an IPv4 address — in the usual dot notation.

Never

3.

Instead of an IP address, you enter an (alternative) host name which is resolvable by DNS.

When running the checks

4.

Use the Hosts with dynamic DNS lookup during monitoring rule set to specify hosts for a dynamic DNS. The result is analogous to option 3, except that the Hostname field is now used for the DNS query.

When running the checks

With the first option Checkmk uses a cache file to avoid repeated DNS requests during the activation of the changes. This cache is very important for speeding up the process, and it also ensures that you can activate a changed configuration even if the DNS fails in an attempt.

The catch is that Checkmk does not immediately notice the change of an IP address in the DNS. That’s why there is a menu entry Host > Update DNS cache in the host properties. This will clear the complete DNS cache and force a new resolution the next time the changes are activated. The corresponding file is located in your site under ~/var/check_mk/ipaddresses.cache. Deleting this file has the same effect as running Update (site) DNS cache.

Checkmk also supports monitoring via IPv6 — even in dual stack. The order of resolution here is given by the operating system settings (/etc/gai.conf). In the Additional IPv4 addresses and Additional IPv6 addresses fields only IP addresses in dot or colon notation are allowed, so alternative DNS names are not permitted.

3.3. Monitoring agents

With the Monitoring agents you decide from which sources data is used for monitoring. The default setting provides for the use of the Checkmk agent. Numerous alternative or additional monitoring options are also provided.

Dialog with properties of a host: Monitoring agents.

In particular, the first Checkmk agent / API integrations entry decides which data to use and which to discard when multiple sources are present.

API integrations if configured, else Checkmk agent

Monitoring data is supplied by API integrations, that is, by special agents or by piggyback from other hosts. If no API integrations are configured, the output from the Checkmk agent is accessed. This is the default value.

Configured API integrations and Checkmk agent

The output from a Checkmk agent is expected. Data supplied by API integrations will also be used if configured.

Configured API integrations, no Checkmk agent

Only data supplied by API integrations will be used for monitoring.

No API integrations, no Checkmk agent

With this setting the host will be monitored via SNMP or agentless only with active checks. Rules for active checks can be found in Setup > Hosts > HTTP, TCP, Email, …​ If you do not define at least one active check, Checkmk will automatically create a PING service.

Note: Selecting Configured API integrations, no Checkmk agent ensures that both of the services Check_MK and Check_MK Discovery always have the OK state. If you have not entered an IP address for the host, it will assume the DOWN state because no ping can be performed. In this case you must use the Setup > Hosts > Host monitoring rules > Host Check Command rule to determine how it is to be verified whether the host is evaluated as UP.

Use the following SNMP setting to configure monitoring via SNMP. After enabling SNMP and selecting the SNMP version, the SNMP credentials field appears, which allows you to specify the SNMP community. However, since this is usually the same for many hosts, it is more advisable to store this setting in the folder. If you do not specify anything, public is automatically assumed.

The last field Piggyback refers to the use of piggyback data, which is piggybacked by other hosts and assigned as agent output to this host. Note that the setting made here must match the settings of the first Checkmk agent / API integrations option, otherwise the absence of expected monitoring data may not be noticed. For this reason, one entry is explicitly Always use and expect piggyback data.

3.4. Custom attributes

In the Custom attributes box, you can display any freely-coded text fields that you have previously created as your own custom attributes. By default, Labels and the two predefined host tag groups Criticality and Networking Segment are present here:

Dialog with the properties of a host: custom attributes.

You can define custom attributes using Setup > Hosts > Custom host attributes.

Menu for creating a custom attribute.

Custom attributes can be, for example, an on-site contact, a branch number, hardware information, inventory numbers, or geographic coordinates. Custom attribute values should primarily help users to keep track, but can also be used in rules and filters when, for example, labels or host tags are too inflexible.

Custom attributes can be assigned to any box in a host’s properties by selecting Topic (topic).

3.5. Management board

The designation Management board stands for separate plug-in cards or extended BIOS functionality (such as Baseboard Management Controller/BMC, Management Engine/ME) for monitoring and managing the hardware in addition to the installed operating system. As well as remote control and remote maintenance functions (e.g. for deploying operating systems), such hardware usually also comes with an IPMI or SNMP interface, via which 'health' values (e.g. temperatures and fan speeds) can be viewed. Since management boards have their own IP address, a separate monitoring would make the assignment more difficult for Checkmk users. Enter the IP address and access method here if the host to be monitored has a management board.

Management board setup.

3.6. Creation / Locking

While most of the details described so far can be edited, the Creation / Locking box contains details that are for informational purposes only at this point.

Display creation and locking information.

The time of creation (Created at) and the creator (Created by) are determined by Checkmk. There are three main choices for the creator name:

  • User name, e.g. cmkadmin: the host was created manually by a user, e.g. in Setup.

  • automation: the host was created by the automation user, e.g. using the REST API.

  • Network scan: The host was found during an automatic network scan.

If the host was created by the automation user, it will be locked by its corresponding site (Locked by). This information can also be used when searching for hosts in Setup.

Locked attributes can no longer be edited in the host’s properties.

3.7. Saving and continuing

When creating or cloning a host, after setting the properties, the next sensible step is Save & go to service configuration. This will take you to the automatic service discovery, which we will explain in the next chapter.

In contrast, Save & go to connection tests takes you to the connection test. This allows you to first test whether you are getting any data at all from the host with the settings you have specified — be it via an agent or whatever you have previously configured. For details on the connection test, see the article on monitoring agents.

3.8. Bulk actions

You may occasionally want to do things like delete, move, edit, or perform a service discovery for a number of hosts at the same time. Checkmk has the so-called 'bulk actions' for this purpose.

You can find these actions in the Hosts menu page of an open folder in the On selected hosts section:

'Hosts' menu with the bulk actions function.

The actions always refer to the hosts that are located directly in the displayed folder — and which have been selected by you. The check boxes in the first column of the host list serve this purpose. If you click the cross in the column title, all hosts will be selected — and deselected by clicking it again.

Here are some notes on the available actions:

  • Delete hosts: Deletes the hosts — after confirming the request.

  • Edit attributes: Changes one or more attributes of the hosts. The attribute is thereby explicitly entered into the hosts.

    Caution: There is a difference between a host inheriting an attribute from a folder and having it explicitly set, as by this action. Why? In the latter case, changing the attributes in the folder will have no effect, since values set directly at the host will always take precedence. For this reason, there is also the following action.

  • Remove explicit attribute settings: Removes explicit attributes from hosts and reinstates inheritance. You would accomplish the same thing by selecting each host individually and unchecking the boxes on the relevant attributes.

    In general, it is a good idea to use as few explicit attributes as possible. If everything is inherited correctly through the folders, this avoids errors and makes it convenient to add new hosts.

  • Remove TLS registration: Removes the registration for encrypted communication via Transport Layer Security (TLS), for instance to reset communication with the Checkmk agent to unencrypted mode.

  • Discover services: Perform a service discovery for multiple hosts simultaneously.

  • Detect network parent hosts: Have parents created by a scan.

  • Move to other folder: Move the hosts to another folder. After selecting this entry, the existing folders are suggested as possible targets.

4. Configuring services

The next step after creating a host is to configure the services to be monitored on it. You can learn all the details covering the automatic discovery and configuration of services in the article Understanding and configuring services. We will describe only the most important points here.

The list of a host’s configured services can be accessed in the following ways:

  • In Setup, via the host list:
    Select Setup > Hosts > Hosts. In the host list, click Icon to display the configured services..

  • In Setup, via the properties of a host:
    Select Setup > Hosts > Hosts. In the host list, click the required host. On the Properties of host page, select Host > Save & go to service configuration. If you select Host > Service configuration from the menu instead, you will also go to the service list — but without saving the host properties.

  • In monitoring, via the list of services of a host:
    Select the Host menu, and in the Setup section select the Service configuration entry. A small gear symbol at the icon shows that this menu entry leads into a page in the Setup. This will take you directly to the service configuration.

Whichever method you choose, the result should look something like this:

List of services found on the host.

The most important actions are available to you in the action bar, others can be found in the Actions menu. Some notes on the possible actions:

  • The Accept all button is usually the best action for a new host. This is also the right choice for existing hosts in which services are found that are not currently being monitored. These can be found in the Undecided services (currently not monitored) section. Accept all adds the missing services, removes vanished services and accepts any host labels found.

  • The Rescan button ensures that fresh, complete data is fetched from the target device. To enable faster page loading, Checkmk works with cached files that are recorded during normal monitoring. For SNMP devices, the button therefore triggers an active search for new check plug-ins and may also find additional services.

  • The Monitor undecided services button transfers the corresponding services to the monitoring, but without transferring the host labels.

  • Remove vanished services removes services that no longer exist. This is useful for services that remember a current state on discovery (e.g., the current state of a switch’s ports or of file systems and their mount points).

  • After each modification an activate changes is required to make them effective.

5. Importing hosts from CSV data

If you want to add a large number of hosts to Checkmk in a single action, you can make the task easier by importing the hosts from a CSV file. This is especially helpful in two usage scenarios:

  • You want to import hosts from another monitoring system that supports exporting in structured data formats. You can create a CSV file from this source system and use this file for the import to the target Checkmk system.

  • You want to create many new hosts at the same time. In such a situation, enter the hosts into an Excel spreadsheet, then simply import this list as a CSV file.

Checkmk is quite flexible when importing CSV data. In the simplest case, in each line of the CSV file you simply have a host name that is resolvable by DNS:

import.csv
myserver01
myserver02

You can also pass additional attributes at the same time during an import. An overview of all possible attributes can be found in the next section. If the CSV file contains the names of the attributes in the first line, Checkmk can even assign these automatically. Checkmk is, as far as possible, tolerant of non-exact spelling (it tries to interpret minor variations). For example, in the following file, Checkmk can automatically map all columns correctly:

import.csv
hostname;ip address;alias;agent;snmp_ds
lnx17.example.com;192.168.178.48;Webserver;cmk-agent;
lnx18.exmpl.com;192.168.178.55;Backupserver;cmk-agent;
switch47-11;;Switch47;no-agent;snmp-v2

The procedure is as follows: Select or create a folder into which the import should take place. Navigate to this folder (Setup > Hosts > …​) and in the Hosts menu select the entry Icon to import multiple hosts in CSV file format. Import hosts via CSV file.

On the following page, use Upload CSV File to upload the file. Alternatively, select Content of CSV File and copy the contents of the file into the text box below the list. You can immediately run an automatic service discovery on the newly imported hosts. To do this, select the Perform automatic service discovery option.

Dialog for entering CSV data for import.

By clicking Icon to accept the input. Upload you will get to the next page:

Dialog to control the CSV data for import.

Selecting a delimiter (Set field delimiter) is not necessary for this example, as the semicolon has already been interpreted correctly. Checkmk automatically recognizes common delimiters, such as tab or semicolon. Select the Has title line option here to recognize the headings. Under Preview you can see a preview of the new table.

If the automatic detection of a column does not work correctly, you can also manually select the attribute that is to be assigned. Use the respective list for this. For the host tags (those entries that start with Tag) the CSV file must contain the internal ID of the tag (here e.g. cmk-agent and not its title Checkmk agent / API integrations visible in the GUI). You can see exactly what the internal IDs of the host tags are in Setup > Hosts > Tags. For the IDs of the predefined (builtin) tags, see the table in the article describing the host tags.

Start the import by clicking Icon to save input. Import. If you have selected the Perform automatic service discovery option, you will land on the Bulk discovery page and should still edit it. Once the service discovery has been completed, all that remains is to activate the changes as usual, after which all the new hosts will be in the monitoring.

5.1. Overview of the attributes for import

AttributeDescription

hostname

Name of the host

Alias

Alias name of the host

Monitored on site

Site on which this host is monitored

IPv4 address

IPv4 address

IPv6 address

IPv6 address

SNMP community

SNMP community

Tag: Criticality

Tag: Criticality

Tag: Networking Segment

Tag: Network segment

Tag: Checkmk agent / API integrations

Tag: Checkmk agent / API integration

Tag: Piggyback

Tag: Piggyback

Tag: SNMP

Tag: SNMP

Tag: IP address family

Tag: IP address family

6. Performing a network scan for folders

6.1. The principle

Checkmk provides the ability to automatically and regularly scan your network — or just parts of it — for (new) hosts. This network scan is set up at the folder level in Checkmk’s host administration. A cronjob runs in the background every minute. This checks all folders to see if another scan is due. The cronjob checks the two settings Scan interval and Time allowed. If this interval has expired and the server is within the time allowed for the scan, the scan will be started. Thus, a newly set up network scan would start within a minute of clicking Save, provided you have not changed the Time allowed.

Once a scan starts, basically three things happen:

  1. Checkmk first determines the IP addresses to be scanned. From the configured address ranges, this removes any addresses that are already being used in any folder in your hosts' configurations.

  2. The identified addresses will now be pinged. If there is a response to this ping on an address, an attempt is made to determine a host name.

  3. Finally the hosts in the folder are created. All hosts with names that already exist on the site will be omitted.

6.2. Setting up a network scan

As mentioned above, the network scan is set up at the folder level. First open Setup > Hosts > Hosts. Then navigate to any folder or stay in the Main. folder. Via the Folder > Properties menu you will find the Network Scan box.

Disabled _Network Scan_ box in the properties of a folder

Enable the network scan using the corresponding checkbox.

Activated but unconfigured _Network Scan_ box

Then, at IP ranges to scan, specify the IP addresses that Checkmk should automatically monitor for you. For this definition you have the choice between single IP addresses, IP ranges and whole networks. We recommend not selecting too large an address range, otherwise the network scan may take a very long time. When choosing a network, we also recommend not to exceed the netmask of /21, which corresponds to 2048 IP addresses. You should not exceed the number of 2048 IP addresses even when selecting via IP-Range. Of course, this can only be a rough recommendation, since your (organizations) network may be able to handle larger address ranges without any problems.

The following IP ranges to exclude option allows you to exclude parts from the address range configured above. This option is also useful for excluding hosts or IP addresses from the network scan that are already known and being monitored. In this way, you can prevent duplicate hosts from being created.

The following two options Scan interval and Time allowed allow you to specify how often the scan should run and at what time you want to allow it.

One of the most important considerations when setting up network scanning is how you want to handle any hosts that are found. The Set criticality host tag option plays a central role here:

  • By default, Do not monitor this host is selected. This specifies that the hosts found are only included in the host administration for the time being. An actual monitoring does not take place. One approach could be to manually move the discovered hosts into your existing host structure — for example using the Move this host to another folder function. After moving, you will then need to customize or remove the Criticality host tag. If you have a large number of hosts, you can use the Remove explicit attribute settings function in Setup > Hosts > Hosts.

  • If you instead select Productive system as the host tag here, the hosts found — if configured appropriately with the Periodic service discovery rule set — will also be included directly in monitoring.

Before using network scanning, also note the following basic considerations:

  • The scan runs via a ping. This also means that devices that can only be monitored via SNMP may be found but will not automatically be monitored because the SNMP credentials will not have been configured.

  • For new Windows hosts, the so-called Echo Request will be deactivated in the firewall unless a corresponding configuration has been defined, e.g. by means of group policies. Such Windows hosts will therefore not respond to the scan and will therefore not be found.

  • Checkmk can only provide you with clean data for the hosts it finds if your network and site are configured accordingly. Conflicting entries in DNS and your Checkmk site can lead to duplication. A duplicate is created when a host with a name (but without an IP address) has already been set up in Checkmk and is now found under a different name via its IP address.

Checkmk provides a convenient way to search in the monitoring environment (in the Monitor menu) and in the configuration environment (in the Setup menu). The results from these searches may differ because the hosts in the monitoring environment are not necessarily the same as in the configuration environment, for example, if you have created a host in Setup without enabling this change, it will not (yet) exist in monitoring.

There is another way to search for hosts in the configuration environment, which has the following advantages:

  • You can search for hosts by a variety of criteria.

  • The hosts which are found are listed on a results page from which you can start the bulk actions, as described earlier in this article.

You can find this search under Setup > Hosts > Hosts on the page of an opened folder in the Display > Search hosts menu. Such a search always starts from the current folder, recursively into all its subfolders. To search globally, simply start the search from the Main folder:

Dialog to search for hosts in a folder.

For the Hostname field, a partial word search (infix search) is used — the entered text is searched, at any position, in the host name. Furthermore, you can limit the search using other attributes. All conditions are linked with AND. So the example from the above screenshot searches for all hosts with my in their names, and which also have the Test system tag.

You start the search with Submit. The result page can be treated almost like a normal folder. This means that here you can use the bulk actions available in the Hosts menu in the On selected hosts section, e.g. to move all found hosts to a specific folder.

You can further customize and refine the search on the results page with Refine search.

8. Renaming hosts

Renaming hosts may seem like a simple task at first glance, on closer inspection, however, it turns out to be a complex operation. The reason is that Checkmk uses the host’s name as a unique key for the host — and indeed in numerous places. This includes file names, log data, rules, dashboards, reports, BI aggregations, and more. A host’s name also appears in URLs.

Checkmk provides two functions in the Setup for cleanly renaming a host in all of its occurrences. You can either rename a single host (in the host’s properties in the Host > Rename menu) or rename multiple hosts in a folder simultaneously (in the Hosts > Rename multiple hosts menu).

Important: When making changes to multiple hosts and in multiple locations, it is always possible for something to go wrong. Therefore, make sure you have an up-to-date backup of your site — before starting the renaming action.

The Bulk renaming of hosts allows simultaneous, systematic name matching for multiple hosts:

Dialog for renaming multiple hosts.

In the Hostname matching field, first optionally specify a regular expression that matches the beginning of the host names you want to rename — so here in the example, all hosts whose names start with lnx. Then add one or more operations with Add renaming. These are applied in sequence to the host names. In the above example, first Drop Domain Suffix truncates everything following the first . from all host names, and then Add Suffix adds the -linuxserver suffix to the names.

Further operations are available, most of which are self-explanatory. Otherwise you can get more information via the inline help.

After starting the renaming procedure with Icon to save input. Bulk rename and confirming the obligatory "Are you sure…​?" question …

Dialog to confirm host renaming.

… completing the process may take a while. During the renaming process the monitoring will be completely stopped! This is necessary to maintain everything in a consistent state. At the end of the process you will get an overview of exactly where renames were performed:

Result of host renaming.

9. REST API requests for hosts and folders

You can also perform many of the actions described in this article using the Checkmk REST API. This is especially interesting if you have many objects to manage on your site and want to automate actions. For example, you can create many hosts using the REST API and avoid the potential errors that can always occur when entering them manually via the GUI.

If the command line, scripts and APIs are not your tools of choice, at this point it is enough to know that this API exists. This gives you a powerful tool to fall back on when you need it — as an alternative to managing via the web interface.

See the REST API article for an introduction to using this API. You can find the reference documentation in your Checkmk site’s web interface. There you can familiarize yourself with the syntax of the requests and the structure of the responses. You can access all important REST API entries in the Checkmk web interface via the navigation bar in the Help > Developer resources menu.

Finally, see the REST API article for Examples which explains how to perform actions on hosts and folders via the command line, such as viewing the folder structure and the hosts in a folder, creating a host in a specific folder, and more.

10. Files and directories

File pathFunction

~/etc/check_mk/conf.d/wato/

In this directory the folder structure in the Setup below the Main folder is mapped by a subdirectory structure. If a folder is created in the GUI, a new directory is also created in the file system. Here Checkmk ensures that the names of the directories are unique and only characters that are allowed in the file system are used. For example, a space is replaced by an underscore.

~/etc/check_mk/conf.d/wato/hosts.mk

Configuration file for all hosts in the Main folder. For hosts in subfolders of Main, there is a file with the same name in each subdirectory.

~/etc/check_mk/conf.d/wato/.wato

This hidden file contains the display name in the GUI (Main) and all other properties of this directory. A file .wato exists in every subdirectory. If a folder is renamed in the GUI, only the title parameter for the display name is changed in this file. The name of the directory in the file system remains unchanged.

~/etc/check_mk/(conf.d|multisite.d)/wato/tags.mk

These files contain the definitions of all host tags.

On this page