Monitoring Amazon Web Services (AWS)

In Checkmk all objects to be monitored are arranged in a hierarchical structure of hosts and services. The concept of hosts does not exist in cloud-based services. However to retain the simplicity and consistency of Checkmk, we still map AWS objects according to the host/service schema.

How that is achieved can best be illustrated by an example: In one region several EC2 instances have been configured. An EC2 is usually assigned to EBS. This constellation looks like this in Checkmk:

AWS provides an HTTP-based API over which monitoring data is also available. Checkmk accesses this API via the agent_aws special agent — which replaces the Checkmk agent — but in contrast this agent runs locally on the Checkmk server.

To enable monitoring via Checkmk, it is best to achieve it by creating a special AWS user under your root account. Log in to AWS as the root user, and navigate in All services to Security, Identity, & Compliance > IAM (Identity and Access Management). Go to Users and create a new user with Add user. As a user name choose, for example, check-mk. It is important that you select the Access key - Programmatic access for Select AWS credential type.

The user you just created should only be used for monitoring by Checkmk and only needs read-only access to AWS. We recommend simply assigning the policy ReadOnlyAccess to this user. To find this policy, first click Attach existing policies directly and then type readonlyaccess in the search box. In the list below the search field, you still have to scroll down quite far, because there are quite a few policies that contain this string.

After completing the user creation an access key will be generated automatically for you. Important: The key’s secret is displayed only once — directly after its creation. Therefore without fail copy the key and save it, for example, in the Checkmk password store. Alternatively specify it in plain text as a rule (see below). For Checkmk you need the Access key ID in addition to the secret. The name of the user (in our example check-mk) does not matter here.

If for some reason you should lose the secret, you can create a new access key for the user and get a new secret:

If you want Checkmk to have read access for the billing information (in order to perform the Costs and Usage global check), you need another policy for your AWS user — a policy you yourself must first define.

Under Security, Identity, & Compliance > IAM > Policies select the Create Policy button. Select from Select a Service > Service > Choose a Service the Billing service. Under Actions tick the Read checkbox. We have to set an additional permission. Add one via the Add additional permissions button. Select from Select a Service > Service > Choose a Service the Cost Explorer Service service. Under Actions again tick the Read checkbox.

Click Review to go to step two. Set BillingViewAccess as the Name, and save it with the Create policy button.

You must now add this new policy to the user. Go again to Security, Identity, & Compliance > IAM > Policies — in the Filter Policies search box look for BillingViewAccess, select this by clicking in the circle to the left, and then go to Policy actions > Attach. Here you will find your check-mk user, select this and confirm it with Attach policy.

Now create a host to monitor AWS in Checkmk. You can assign the host name as you wish. Important: Because AWS as a service has no IP address or DNS name (access is granted by the special agent itself), you need to set the IP address family to No IP.

AWS cannot be queried through the regular Checkmk agent, so next set up the AWS special agent. To do so, under Setup > Agents > VM, Cloud, Container > Amazon Web Services (AWS) add a rule whose conditions apply only to the just-created AWS host.

In the actual content of the rule, you will first find the information for the login. Here enter the 'Access key ID' for the newly-created AWS user check-mk. Also choose here if you need a proxy to fetch the data and which global data you want to monitor, i.e., those that are independent of a region. That is currently only the data relating to the costs:

In the image above you can also see the option Use STS AssumeRole to assume a different IAM role. If you do use different accounts at AWS, you may use a single monitoring user to monitor the other accounts, too.

But the really interesting data is assigned to regions. Therefore here select your AWS region(s):

Under Services by region to monitor you specify which information you want to retrieve from these regions. At default all AWS services and the monitoring of their limits are activated. In the following image are all but one deactivated to get a better overview:

You now can restrict the fetched data per web service or globally with Restrict monitoring services by one of these AWS tags. The global restriction will be overwritten, if you restrict by web service. Also you not only have the option to restrict by AWS tags but additionally to specify the explicit names:

Don’t forget to assign the special agent to the previously created host by entering that host name in Conditions > Explicit hosts.

Now switch to the service discovery of the newly created AWS host in Checkmk, where Checkmk should now find several services. After you add the services and after an activating changes it will look something like this in monitoring:

Services that are assigned to EC2 instances are not assigned to the AWS host, but to so-called piggyback hosts. This works in such a way that data retrieved from the AWS host is distributed to these piggyback hosts, which operate without their own monitoring agents. A piggyback host is assigned to each EC2 instance.

For the naming of these piggyback hosts, you can choose between two schemes when configuring the special agent. On the one hand, you can name the hosts according to their private IP DNS name or you can choose the somewhat longer but unique naming according to IP, region and instance ID. The latter variant is our default setting as of Checkmk 2.2.0. The variant without region and instance ID is still offered only for compatibility reasons. Such a piggyback host could thus be named 172.23.1.123-ap-northeast-2-i-0b16121900a32960c, for example. Either create these hosts manually or - if possible - leave this task to the dynamic host management.

The services for the ELB are also assigned to piggyback hosts. The names correspond to their DNS names.

With Checkmk you can monitor the traffic of each of your S3 buckets. In Checkmk you simply have to activate the option Request metrics below Simple Storage Service (S3).

In AWS, a little more work is required. Here you still need to set up these Request metrics for the buckets you want to monitor. AWS describes how this works in detail in the article Creating a CloudWatch metrics configuration for all the objects in your bucket. During setup in AWS, you will be asked to create a filter. You have to name this filter EntireBucket so that it is recognized by Checkmk. Any filter with a different name will be ignored by Checkmk. So you are free to define other filters for this bucket without affecting the functionality in Checkmk.

How you choose the so-called (filter) Scope in AWS is also up to you. In most cases, however, it will make sense to include all objects in the bucket in the filter.

After setting up the Request metrics, it will take a few minutes before any metrics are stored at all. AWS specifies this time as 15 minutes.

Important: As long as the graphs inside the S3 console are still empty, nothing will arrive in Checkmk via the special agent either. Only when metrics have been recorded, Checkmk can create the corresponding services. If necessary, perform again a service discovery on the AWS host.

Some web services of AWS do have limits and Checkmk is able to monitor them. Here some examples:

As soon as such a check plug-in creates services and checks them later on, the special agent will always fetch all elements of the web service. Only in this way Checkmk is able to reasonably calculate the current workload at these limits and check the thresholds. That’s also the case even if you restrict the fetched data by some tags or names in the configuration.

The checking of the limits is activated by default for each monitored web service. If you want to restrict the fetched data in the special agent rule to limit the amount of transferred data, you need also to turn off the limits.

The other web services in AWS are assigned as follows: