Rules

As an example, let’s take the configuration of the thresholds for WARN and CRIT in the monitoring of file systems. With a data base-oriented configuration, for each file system one would enter a line into a table:

This is relatively straightforward — but only because the table in this example is short. In practice there tend to be hundreds or thousands of file systems. Tools like copy & paste, and bulk operations can simplify the work, but the basic problem remains — how can you identify and implement a standard policy? What is the general rule? How should thresholds for future hosts be preset?

A rules-based configuration however consists of the policy! We will replace the logic of the above table with a set of four rules. If we assume that myserver001 is a test system, and that in each case the first relevant rule applies to every file system, the result will be the same thresholds as in the table above:

Granted, for only two hosts that doesn’t achieve much, but with only a few more hosts it can quickly make quite a big difference. The advantages of the rules-based configuration are obvious:

In summary, then: less work — more quality! For this reason, with Checkmk you will find an abundance of rules for customizing hosts and services — such as thresholds, monitoring settings, responsibilities, notifications, agent configuration and many more.

Within Setup Checkmk organizes rules in rule sets. Every rule set has the task of defining a specific parameter for hosts or services. Checkmk contains more than 700 rule sets! Here are some examples:

Every rule set is responsible either for hosts or for services — never for both. If a parameter can be defined for hosts as well as services, there is a pair of applicable rules — e.g., Normal check interval for host checks and Normal check interval for services checks.

A few rule sets, strictly-speaking, don’t actually define parameters, rather they create services. An example are the rules for active checks, which can be found at Setup > Services > HTTP, TCP, Email, …​. With these you can, e.g., set up an HTTP check for specific hosts. These rules are classified as host rules — due to the fact that if such a check exists on a host it is deemed to be a property of the host.

Further, there are rule sets that control the service discovery. With these you can, for example, via Windows service discovery define for which Windows services automatic checks should be created if they are found on a system. These are also host rules.

The bulk of the rule sets determine parameters for specific check plug-ins. An example is Network interfaces and switch ports. The settings in these rules are tailored very specifically to their appropriate plug-in. Such rule sets fundamentally only find use with those services that are based on this plug-in. In case you are uncertain which rule set is responsible for which services, then you can best find out by navigating directly via the service to the relevant rule. How to do this will be explained later.

One thing we have so far not mentioned: In the above example there is a rule for all test systems. Where is it actually defined that a host is a test system?

In Checkmk, something like test system is known as a host tag. You can see which tags are available via Setup > Hosts > Tags. Some tags are already predefined — for example, for a Test system defined in the Criticality group.

Applying tags to hosts is done either explicitly in the properties of the host, or through inheritance in the folder hierarchy. How to do this is explained in the article on hosts. How to create your own tags, and what the predefined tags are about will be explained in the article on host tags.

If you wish to create a new rule that defines a parameter for one or more hosts, there are several ways to this end. The direct way is via the corresponding group in the setup menu, in this case Setup > Hosts > Host monitoring rules:

In the following view, all rule sets relevant for host monitoring are displayed. The numbers following the names of these rule sets show the number of rules that have already been defined:

However, you can reach your goal somewhat faster via the search field. To do this, of course, you need to know approximately what the rule set is called. Here is the result of a search for host checks as an example.

Another way is via the menu item Hosts > Effective parameters in the properties of an existing host in the Setup or via the icon in the list of hosts of a folder.

There you will find not only all the rule sets that affect the host, but also the parameter currently effective for this host. In the example of Host check command no rule applies for the shown host, and it is therefore set to the Smart PING (only with Checkmk Micro Core) default value of the commercial editions. In Checkmk Raw the default value is PING (active check with ICMP echo request).

Click on Host check command in order to see the complete rule set.

If a rule already exists, instead of the Default value the number of the rule defining this parameter appears.

Clicking on this takes you directly to the rule.

The path to the rule sets for services is similar. The general access is via the Setup menu, in this case Setup > Services > Service monitoring rules or, more appropriately via the search field.

If you are not yet very experienced with the names of the rule sets, then the path via the service is simpler. Similarly to the hosts, there is also a page in which all of a service’s parameters are shown and where you have the possibility of directly accessing the applicable rule sets. You can access this parameter page with the icon in a host’s list of services in the Setup. The icon takes you directly to the rule set that defines the parameter for the check plug-in for this service.

By the way — the icon for the parameter page is also found in the monitoring in every service’s action menu:

In the Setup menu you will also find an entry for Enforced Services. As the name suggests, you can use these rule sets to force services to be created on your hosts. Details can be found in the article about services. A small number of rule sets — such as Simple checks for BIOS/Hardware errors — can only be found under the enforced services. These are services which do not result from the service discovery, but are created manually by you.

In each of the aforementioned lists of rule sets — whether in the Host monitoring rules or the Service monitoring rules — you can use Related > Used rulesets in the menu bar, to display only the rule sets in which you have defined at least one rule. This is often a convenient way to get started if you want to make adjustments to your existing rules. Incidentally, some of the rules will have been generated by default when creating the Checkmk site and are a part of the sample configuration. These are also displayed here.

Monitoring is a complex matter. It can happen that there are rules which do not match a single host or service — either because you have made a mistake or because the matching hosts and services have disappeared. Such ineffective rules can be found in the aforementioned rule set listings via Related > Ineffective rulesets in the menu bar.

Checkmk is under constant development. Occasionally things are standardized and it can occur that some rule sets are replaced by others. If you have such rule sets in use, the easiest way to find them is do a rule search. Open it via Setup > General > Rule search. Then click in the menu bar on Rules > Refine search, select Search for deprecated rulesets as the option for Deprecated and select Search for rule sets that have rules configured as the option for Used. After an additional click on Search you get the desired overview.

When you access a rule set for a host or service in Setup, Checkmk will show you this rule set in the analysis mode. You can get there by clicking on the icon in the action menu in the Setup in the host or service list. The following Effective parameters of page shows the list of rules that apply to the host/service. To go to the analysis mode, click on the name of a rule set for which at least one rule exists, i.e. a set which is not set to the Default value:

This mode has two features. Firstly, a second button for setting rules appears — Add rule for current host bzw. Add rule for current host and service.

With this you can create a new rule which has the appropriate current host or service already preselected. You can create an exceptional rule very easily and directly in this way. Secondly, a ‘traffic light’ icon appears in every line, the color of which shows whether and/or how this rule affects the current host, or respectively, service. The following conditions are possible:

In the last condition — the rule is a partial match — can only occur for rule sets in which a rule can define multiple parameters by selecting individual check boxes. Theoretically, every parameter for another rule can also be set individually here. More on this later.

Everything in the first block, Rule Properties, is optional, and serves primarily for documentation:

The second section is different for each rule, but always specifies what should be done. The following illustration shows a widely-used type of rule (DB2 Tablespaces). You can use checkboxes to determine which individual parameters the rule should define. As described above, Checkmk determines which rule defines each individual parameter separately. The rule from the illustration therefore only defines the one value and leaves all other settings unaffected:

You can also control the values in this and other rules on a time/calendar basis. For example, you can set threshold values so that tablespace usage during business hours differs from that on weekends.

First click the Enable timespecific parameters button and then click Add new element, you will see the time-dependent options:

Now select a time period in the Match only during time period list, and then select the parameters for which this time period should apply.

Some of the rule sets do not set a parameter, but only decide which hosts are in and which are not. An example is the rule set Hosts to be monitored, whose parameter range looks like this:

By selecting one of the two available values, you decide what to do with the affected hosts. Selecting Positive match (Add matching hosts to the set) will add the affected hosts to the set of hosts to be monitored. Selecting Negative match (Exclude matching hosts from the set) removes the affected hosts from the monitoring. The Positive match or Negative match refers to the content of the current rule. It is not an additional filter criterion for selecting hosts. You filter the set of affected hosts exclusively with the following Conditions.

In the previous section, you defined how all those hosts or services that are affected by your rule are to be processed. In the third section Conditions you now define which hosts or services are to be acted on by the rule — and thus its effects. There are different types of conditions that must all be fulfilled for the rule to take effect. The conditions are therefore logically AND-linked:

In the introduction to the concept of rules, you saw that the first rule that applies always determines the final outcome. This is not the whole truth. There are a total of three different types of evaluation:

The information on how the rule is evaluated is displayed for each rule set directly below the menu bar:

Now, how will it evaluated concretely if one has created several rules that are to be applied to several hosts? To illustrate this, let’s take a simple example:

Let’s say you have three hosts and you want to set different periodically-repeated notifications for each of them (and also for all hosts added in the future) with the Periodic notifications during host problems rule:

If you now activate your configuration, Checkmk runs through the chain of rules from top to bottom. This results in the following evaluation:

Please note here: Since 'The first matching rule defines the parameter' applies to this rule set, the processing of the rule chain is always terminated after the first match. The order of the rules is therefore decisive for the result! This becomes apparent when the order of the rules is changed and rules B and C are swapped:

If the rule chain is now run through again from top to bottom for the individual hosts, the result also changes: Rule C now applies not only to Host-3, but also to Host-2, so that the notification for both hosts takes place every 30 minutes. This completes the processing for both hosts. Although Rule B would be relevant for Host-2, and was even written for this host, it will no longer be evaluated and applied. In the analysis mode, the process will then look like this:

By combining the various settings explained in this article — keeping in mind the correct processing order — you can use them to build complex rule chains for entire host complexes.