Monitoring via SNMP

Routers, switches, firewalls, printers, appliances, UPSs, hardware sensors, and many other devices do not allow the installation of a Checkmk agent. They do however already have a built-in interface for monitoring provided by their manufacturer — an SNMP agent. This agent can be accessed via the Simple Network Management Protocol (SNMP). Checkmk uses SNMP to monitor such devices. The benefit for you is that setting up the monitoring is very easy.

There are also SNMP agents for Windows and Linux, however using these instead of the Checkmk agent is not recommended. SNMP is not very performant, so using this for monitoring generally means that the Checkmk server needs more CPU and memory per host than when working with its own agent. In addition the data provided via SNMP are incomplete. In some cases, however, monitoring via SNMP in addition to the Checkmk agent can be useful. See the chapter on SNMP and Checkmk agent for more on this topic.

However, if there is no Checkmk agent plug-in for monitoring a particular hardware or software component (e.g. a RAID controller), but the component has an SNMP interface, you can of course collect additional monitoring data via SNMP. In such a case, make sure that the query intervals are long enough.

Monitoring SNMP devices with Checkmk is very easy. If you just want to get started quickly with SNMP, you will probably need to read the short section on SNMP in the Beginner’s guide. This article goes into much more depth, and shows you all of the details and specific scenarios when SNMP monitoring with Checkmk.

The SNMP protocol is available in different versions. These protocols are all incompatible with each other, and so the monitoring system and the monitored device must always consistently use the same protocol version. Checkmk supports versions v1, v2c, and v3. In practice, an estimated 99 % of installations use v2c. Here is an overview of all of the relevant versions of SNMP:

Checkmk uses active requests for SNMP monitoring – a pull method. Checkmk sends a UDP packet (port 161) containing an SNMP request to the device requesting the provision of specific data. The device then responds with a UDP packet containing the response data (or an error message).

SNMP has a second process: SNMP traps. These are spontaneous push messages sent by devices at configured addresses via UDP (Port 162) in push mode. Traps have many disadvantages over active requests, which is the reason they are not very important for monitoring. Some of the disadvantages are:

However if you want or need to work with traps, the Event Console provides a solution. This can receive traps and generate events from them.

The first step is to prepare the device. Each device that supports SNMP has its own configuration mask somewhere in its configuration. Make the following settings in this configuration mask:

The Community is a kind of password, except that there is no user name for SNMP. There is a convention that the community is public. This is the default for many devices — and also for Checkmk. Of course you can argue that this is insecure and that you should specify another community. This certainly makes sense, but you should know that SNMP transmits the community in plain text (except for SNMP Version 3).Anyone who can listen to packets can therefore very easily identify the community. On the other hand you have limited access to read-only access, and most of the information that can be retrieved via SNMP is not very critical.

Furthermore, the use of different communities per device is very cumbersome to handle. After all, these must not only be maintained in the devices, but also in the monitoring system. That is why in practice users usually use the same community everywhere — or at least everywhere within a region, department, data center, etc.

Tip: If you want to increase the security even without SNMP version 3, it makes sense to extend the network concept so that you put the traffic with the management services, and thus also SNMP, in a separate management VLAN and secure that access with the firewall.

Add the monitored devices as hosts in Checkmk in the usual way. If you have chosen your folder structure so that only one folder contains SNMP devices, you can make the other settings directly in that folder. This makes it easier to add additional hosts later, and also avoids errors.

Now in the properties of the host (or folder), in the Monitoring agents box set Checkmk agent / API integrations to No API integrations, no Checkmk agent.

In the same box, also activate SNMP and as the SNMP protocol select SNMP v2 or v3. The selection of protocol version 1 is an makeshift solution only for very old devices. You should use this only if you know that v2 is really not supported, or when the implementation for the device is defective (in practice, only in isolated cases). Above all, SNMP version 1 is very slow because it does not support bulk accesses. This difference is very significant.

The third and final setting is called SNMP credentials. Here again a choice of the protocol version is necessary, since v2c and v3 differ here. We will discuss version 3 below. If you do not have very high security requirements, you will be well-served by v2c — or you can place the SNMP communication in a management VLAN and thus secure it. SNMP v2c requires the entry of the Community as discussed above.

There is an alternative way to configure the SNMP credentials, if you can not easily pass it through your folder structure: the Setup > Agents > SNMP rules > SNMP credentials of monitored hosts rule set. This will allow you to assign the credentials based on host tags, labels and similar properties. The principle is that a community that is set directly at the host or folder always takes precedence over the rules.

When you have finished with the settings, you can make a short detour via the diagnostics page. To do this save with the Save & go to connection test action bar button. Here is an example of the diagnostics for a switch. Various protocol versions of SNMP are tried simultaneously, namely:

A normal, modern device should respond to all four variants with the same data — however this may be limited depending on the configuration. The result will look like this:

The four information outputs are described here:

It actually seems as if any conceivable mistake that can theoretically be made when implementing SNMP has already been made by some manufacturer at some point. And so there are devices with which SNMP works reasonably well, but certain parts of the protocol do not, or have been incorrectly implemented.

If the problems occur with the commercial editions, one reason may be that the more performant inline SNMP implementation, which is activated there by default, relies more heavily on compliance with standards than snmpget. If devices do not respond at all or not reliably, it sometimes helps to switch off inline SNMP for affected devices and thus activate the somewhat more robust and significantly slower snmpget.

For tests on the command line, the command cmk has for some options the additional --snmp-backend option, which accepts inline (use of inline SNMP), classic (use of snmpget) or stored-walk (use of a stored SNMP walk) as parameters. If the test on the command line was successful, you can use the Hosts not using Inline-SNMP rule set to specify the hosts that should not use inline SNMP permanently.

You have included an SNMP device in the monitoring but Checkmk recognizes only the SNMP Info and SNMP Uptime services and the interfaces. This can be due to a number of causes:

If the ping works, but none of the SNMP protocol versions work, there can be several possible causes:

By default SNMP is unencrypted and is therefore very poorly-authenticated by a Community transmitted as plain-text over the network. This level may still be sufficient for a local, isolated network, as here monitoring is limited to accessing read-only operations.

If you still want a higher level of security you will need SNMP version 3. This provides the option of encryption and genuine authentication. For this however a corresponding configuration is also necessary.

SNMP v3 recognizes various levels of security:

The required setting in Checkmk is made in the same place where you also defined the Community — either under the host properties or the SNMP credentials of monitored hosts rule. There, instead of SNMP Community, select one of the three levels of v3 and configure the necessary values:

SNMP v3 introduces the concept of Contexts. A device can show different information at one and the same point in the SNMP tree — depending on which Context ID is given in the query.

If you have a device that works with such contexts, you will need two settings in Checkmk:

Luckily there are very few situations in which you have to work with contexts, because unfortunately it is not possible for the monitoring to recognize them automatically. A manual configuration of the contexts is always necessary.

Performance always plays a role — especially in environments with many hosts — and monitoring with SNMP consumes more CPU and memory than with Checkmk agents.

While Checkmk Raw makes SNMP requests in the classic way via the snmpget or snmpbulkwalk commands, the commercial editions have a built-in SNMP engine that performs SNMP requests very efficiently without generating any extra processes. With this, CPU load for SNMP processing is approximately halved. The shorter polling times also reduce the number of Checkmk processes needed concurrently, and thus the memory usage.

If your resources reach their limits, or if it takes more than 60 seconds to poll a single device, you can reduce the interval at which Checkmk queries the host(s). With the Normal check interval for service checks rule set, which you apply specifically to the Checkmk-Services of hosts, you can extend the general interval of one minute to, for example, 2 or 5 minutes.

Especially for SNMP checks, there is also the Check intervals for SNMP checks rule set. This allows you to lower the interval for individual check plug-ins. It is important to know that you can never set the interval to faster than the interval for general monitoring by the Checkmk service.

Overall, however, we recommend that the monitoring be designed so that the standard interval of one minute can be maintained, and only increased in exceptional cases for individual hosts or checks.

By default Checkmk expects a response in less than one-second for an SNMP request. It also tries a total of three times before giving up. For devices that respond very slowly, or that can only be reached over a very slow network, it may be necessary to change these parameters. You do this through the Timing settings for SNMP access rule set:

Please note that these settings apply to an individual SNMP request. The complete process of monitoring a host consists of many separate requests. The total timeout is therefore a multiple of the settings specified here.

By default SNMP transmits 10 responses in one packet per GetBulk request. Try the Bulk walk: Number of OIDs per bulk experimental rule set to see if a higher value performs better. However this will only be the case when large tables are transferred to the host — e.g., if it is a switch with many ports.

SNMP always fills the packets up to the specified number, including any records following the actual required ones. And if only a few of these records are really needed, extra data is transferred uselessly and the overhead increases.

On the other hand, in practice it may occasionally occur that devices with the default value of 10 OIDs per bulk packet can have problems. In such a case it can be useful to reduce the number.

Checkmk normally works by always getting the information on all switch ports, even though not all are actually being monitored. This is a good thing anyway, since this is normally faster because single queries cannot be done with the efficient bulk queries. In addition, from our point of view, it is always advisable to monitor all ports in order to find faulty ports or cables with high error rates. If ports are not reliably UP, you can also flag the link status DOWN as being OK.

However, there are isolated cases where switches have very many ports, and which for some reason respond very slowly, or process SNMP very inefficiently, so that it is no longer possible to monitor with complete retrieval of all port information.

For such cases, there is the Bulk walk: Limit SNMP OID Ranges rule set. This allows you to statically limit the list of queried data (e.g., ports). In the rule’s value, for each particular check plug-in you specify which indexes of the respective table are to be fetched.

The usual check type for switch ports is called SNMP interface check with 64 bit counters (using v2c). The following example shows a setting in which only the first two ports are fetched via SNMP:

Note: This filtering is then in effect before the service discovery and monitoring. Depending on the Network interface and switch port discovery setting, this does not automatically mean that these two ports will actually be monitored.

The Checkmk SNMP engine has a very handy feature — you can have a monitored device write a complete snapshot of all its SNMP data to a file, an SNMP walk. You can use this file later to simulate monitoring the device on another Checkmk server, even if this other server has no actual network connection to the device.

We use this feature very intensively, for example, when our support team is developing new check plug-ins for our customers. Our developers therefore do not need access to your devices — just an SNMP walk.

You can create an SNMP walk directly from the GUI. The function can be found in the context menu of the Checkmk service of the hosts and also in the menu of the hosts (entry Download SNMP walk):

The creation of the walk takes a few seconds in the best case, but a few minutes are not uncommon. When the build is done you can download the created file via the Result line.

Alternatively, you can also create walks from the command line. Log on to the site from which the device is being monitored. The creation of the walk is simply done with the cmk --snmpwalk command and the specified host (which must be configured in monitoring):

Also use the -v switch to see more detailed output on the progress:

The file will be placed in the var/check_mk/snmpwalks directory, where it simply carries the name of the host. It is a text file. If you are curious you can view this — e.g., with less — and quit the program with the Q key:

The command cmk --snmpwalk has some more useful options:

If you want to use this walk on a different (or the same) Checkmk site for a simulation, then save the walk file with the name of the host on this site under var/check_mk/snmpwalks. Make sure that the site user is owner of the file and permissions are set to 0600 (only the owner is allowed to read and write).

Now create a rule in the Simulating SNMP by using a stored SNMP walk rule set that accesses the affected host(s).

From now on, only the saved file will be used to monitor the host. There is now no network access to the host — except the ping for the host check, and possibly any configured active checks. You can simply redirect these to the Checkmk server by giving the IP address 127.0.0.1 to the hosts.